-- Logs begin at Thu 2022-06-23 10:50:16 UTC, end at Thu 2022-06-23 12:05:04 UTC. -- Jun 23 11:24:23.077071 dv-cinder-820994-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 23 11:24:23.087440 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 23 11:24:23.089439 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 23 11:24:23 2022] *** Jun 23 11:24:23.089439 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 23 11:24:23.089439 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 23 11:24:23.089439 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: nodename: dv-cinder-820994-2-iscsi Jun 23 11:24:23.089439 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: machine: x86_64 Jun 23 11:24:23.089439 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: clock source: unix Jun 23 11:24:23.089439 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: pcre jit disabled Jun 23 11:24:23.089439 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: detected number of CPU cores: 4 Jun 23 11:24:23.089439 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: current working directory: / Jun 23 11:24:23.089439 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: detected binary path: /usr/bin/uwsgi-core Jun 23 11:24:23.089994 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: your processes number limit is 47680 Jun 23 11:24:23.089994 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: your memory page size is 4096 bytes Jun 23 11:24:23.089994 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: detected max file descriptor number: 2048 Jun 23 11:24:23.089994 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: lock engine: pthread robust mutexes Jun 23 11:24:23.089994 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: thunder lock: enabled Jun 23 11:24:23.089994 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 23 11:24:23.089994 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 23 11:24:23.134364 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: Python main interpreter initialized at 0x55e50244f9c0 Jun 23 11:24:23.134364 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: python threads support enabled Jun 23 11:24:23.134364 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: your server socket listen backlog is limited to 100 connections Jun 23 11:24:23.134364 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: your mercy for graceful operations on workers is 90 seconds Jun 23 11:24:23.134707 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: mapped 403077 bytes (393 KB) for 2 cores Jun 23 11:24:23.134824 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: *** Operational MODE: preforking *** Jun 23 11:24:23.135094 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: *** uWSGI is running in multiple interpreter mode *** Jun 23 11:24:23.135094 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: spawned uWSGI master process (pid: 80244) Jun 23 11:24:23.135285 dv-cinder-820994-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 23 11:24:23.135829 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: spawned uWSGI worker 1 (pid: 80255, cores: 1) Jun 23 11:24:23.136165 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: spawned uWSGI worker 2 (pid: 80256, cores: 1) Jun 23 11:24:23.137815 dv-cinder-820994-2-iscsi devstack@keystone.service[80244]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 23 11:24:28.671419 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 11:24:28.671419 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: from cryptography.utils import int_from_bytes Jun 23 11:24:28.672198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 11:24:28.672198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: from cryptography.utils import int_from_bytes Jun 23 11:24:28.673005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 11:24:28.673005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: from cryptography.utils import int_from_bytes Jun 23 11:24:28.673472 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 11:24:28.673472 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: from cryptography.utils import int_from_bytes Jun 23 11:24:28.721221 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 11:24:28.721413 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 11:24:28.721601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] command line args: [] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 11:24:28.721863 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 11:24:28.722037 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ================================================================================ {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 11:24:28.722293 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] admin_token = **** {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.722565 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] config_dir = [] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.722657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 11:24:28.722743 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.722820 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 11:24:28.722894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] config_source = [] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.723009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] command line args: [] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 11:24:28.723132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] debug = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.723252 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.723511 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 11:24:28.723511 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ================================================================================ {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 11:24:28.723628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.723691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] admin_token = **** {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.723754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.723966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] config_dir = [] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.724056 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.724122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.724271 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.724350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] config_source = [] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.724415 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] debug = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.724484 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] list_limit = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.724628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] log_config_append = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.724762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.724919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.725062 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.725211 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] log_dir = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.725346 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.725434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] log_file = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.725605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.725735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.725858 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.725945 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.726016 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] list_limit = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.726079 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.726153 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] log_config_append = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.726222 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.726288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.726364 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.726435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] log_dir = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.726581 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.726670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] log_file = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.726747 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.726872 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.726959 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.727031 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.727146 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.727220 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.727339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.727423 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.727423 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.727547 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.727667 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.727748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.727819 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.727944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.728072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.728151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.728307 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.728387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.728542 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.728776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.728894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.729020 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.729139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] publish_errors = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.729264 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.729385 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.729504 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.729623 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.729623 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.729847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.730012 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.730129 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.730241 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.730359 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] standard_threads = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.730479 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] publish_errors = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.730629 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.730767 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.730882 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.731004 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.731119 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.731238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.731351 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] use_journal = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.731462 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.731580 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] use_json = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.731580 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] use_stderr = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.731806 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.731918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] use_syslog = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.732036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] standard_threads = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.732157 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.732270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.732395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.732509 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.732622 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.732740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.732740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] use_journal = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.732952 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.733138 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] use_json = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.733260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.733375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] use_stderr = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.733375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] use_syslog = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.733593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.733706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:24:28.733829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.733944 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.734057 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.734169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.734282 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.734400 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.734522 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.734678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.734794 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.734906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.735019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.735131 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.735254 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.735353 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.735467 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.735580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.735699 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.735812 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.735932 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.735932 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.736144 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.736144 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.736355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.736467 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.736594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.736750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.736880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.736995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.737108 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.737232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.737346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.737460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.737572 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.737684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.737796 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.737920 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.738042 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.738155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.738267 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.738379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.738493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.738718 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.738843 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.738958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.739071 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.739189 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.739312 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.739435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.739556 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.739670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.739782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.739894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.740007 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.740127 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.740241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.740364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.740478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.740591 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.740709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.740831 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.740951 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.741064 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.741176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.741289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.741401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.741521 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.741636 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.741755 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.741875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.741972 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.742071 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.742176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.742286 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.742388 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.742483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.742653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.742768 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.742873 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.742971 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.743093 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.743213 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.743327 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.743446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.743569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.743685 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.743795 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.743899 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.744002 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.744123 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.744246 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.744362 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.744487 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.744602 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.744717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.744830 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.744943 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.745068 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.745184 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.745295 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.745409 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.745522 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.745636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.745761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.745761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.745983 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.746096 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.746209 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.746321 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.746445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.746615 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.746734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.746850 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.746963 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.747076 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.747198 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.747198 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.747422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.747536 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.747649 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.747649 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.747877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.747999 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.748115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.748228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.748341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.748341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.748569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.748569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.748794 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.748794 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.749007 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.749120 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.749246 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.749361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.749481 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.749601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.749716 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.749716 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.750010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.750127 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.750241 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.750366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.750482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.750607 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.751218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.751319 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.751432 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.751546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.751659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.751781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.751905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.752033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.752150 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.752264 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.752377 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.752377 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.752589 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.752711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.752826 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.752826 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.753039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.753039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.753254 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.753376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.753491 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.753605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.753718 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.753831 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.753945 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.753945 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.754175 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.754296 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.754416 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.754572 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.754690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.754819 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.754936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.755057 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.755172 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.755293 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.755407 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.755529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.755644 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.755757 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.755870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.755984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.756096 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.756218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.756333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.756446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.756559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.756672 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.756784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.756906 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.757021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.757134 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.757247 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.757367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] auth.external = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.757482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.757604 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.757719 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.757832 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.757946 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.758060 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.758173 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.758294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] auth.password = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.758410 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.758591 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] auth.token = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.758739 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.758869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.758986 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] auth.external = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.759100 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.759221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.759344 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.759461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.759577 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.759698 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.759814 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.759928 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] auth.password = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.760050 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.760166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] auth.token = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.760284 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.760400 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.760515 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] credential.caching = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.760629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.760752 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.760868 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.760982 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.761096 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.761210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.761333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.761458 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.761575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.761689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.761803 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.761925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.762042 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] credential.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.762164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.762280 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.762394 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.762508 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.762651 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.762766 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.762893 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.763016 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.763132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.763253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.763368 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.763503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.763621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.763735 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.763855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.763971 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.764085 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.764274 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.764392 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] federation.caching = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.764507 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.764622 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.764737 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.764851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.764974 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.765102 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.765218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.765333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.765448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.765563 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.765691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.765808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.765931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] federation.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.766046 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.766161 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.766275 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.766997 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.767125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.767242 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.767357 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.767472 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.767587 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.767715 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.767835 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] identity.caching = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.767957 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.768073 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.768188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.768303 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.768426 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.768543 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.768659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.768659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.768875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.768996 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.768996 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.769223 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.769337 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.769452 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.769566 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.769681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.769804 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.769927 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.770050 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.770166 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.770287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.770418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.770588 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.770708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.770823 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.770939 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.771054 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.771168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.771295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.771412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.771527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.771642 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.771756 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.771870 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.771994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.772111 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.772226 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.772340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.772454 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.772569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.772692 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.772808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.772929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.773046 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.773161 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.773285 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.773285 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.773502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.773623 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.773739 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.773854 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.773976 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.774093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.774208 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.774322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.774444 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.774582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.774711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.774830 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.774945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.775060 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.775175 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.775290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.775412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.775529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.775643 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.775764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.775880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.775994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.776117 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.776235 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.776349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.776464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.776578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.776693 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.776815 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.776932 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.777047 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.777161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.777276 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.777390 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.777513 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.777630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.777751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.777873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.777989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.778104 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.778228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.778345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.778460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.778587 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.778714 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.778831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.778955 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.779071 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.779187 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.779302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.779416 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.779531 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.779654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.779771 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.779885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.780000 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.user = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.780114 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.780229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.780355 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.780472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.780588 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.780703 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.780817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.780932 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.781055 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.781172 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.781286 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.781402 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.781517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.781631 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.781753 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.781869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.781984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.782098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.782213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.782327 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.782451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.782616 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.782733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.782848 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.782963 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.783077 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.783204 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.783321 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.783435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.783550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.783664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.783778 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.783901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.784017 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.784247 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.784247 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.784470 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.784586 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.784586 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.784814 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.784929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.785044 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.785044 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.785260 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.785260 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.785485 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.785485 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.785701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.785816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.785931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.785931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.786156 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.786270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.786385 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.786499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.786692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.786824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.786942 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.787056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.787171 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.787171 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.787386 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.787575 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.787695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.787810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.787924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.788038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.788162 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.788279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.788393 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.788508 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.788622 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.788738 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.788861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.788984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] resource.caching = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.789101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.789216 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.789330 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.789445 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.789569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.789685 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.789801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.789916 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.790030 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] resource.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.790145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.790269 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.790389 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.790504 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.790650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.790765 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.790882 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.791009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.791127 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.791242 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.791357 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] role.caching = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.791472 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.791587 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] role.driver = sql {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.791710 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.791710 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.791928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.792042 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.792157 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.792272 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] role.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.792399 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.792517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] role.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.792632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.792761 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.792873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.792971 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.793077 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.793177 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.793275 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.793370 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.793466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.793561 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.793666 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.793765 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.793862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.793957 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.794054 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.794158 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.794264 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.794362 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.794465 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.794599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.794702 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.794798 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.794798 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.804950 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.805036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.805109 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.805228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.805315 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.805315 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.805494 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.805570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.805641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.805710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.805789 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.805862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.805933 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.806010 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.806081 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.806151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.806228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.806301 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.806377 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.806449 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.806519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.806618 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.806701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.806775 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.806846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.806916 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.806985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.807055 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.807132 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.807204 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.807273 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.807342 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.807412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.807482 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.807559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.807631 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.807700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.807770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] token.caching = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.807840 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.807910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.807987 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.807987 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.808116 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.808116 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.808242 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] token.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.808312 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.808390 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.808461 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.808531 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.808600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.808670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.808740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.808817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.808889 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.808958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.809028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.809098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.809168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.809245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.809318 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.809388 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.809458 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.809527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.809648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.809734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.809808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.809878 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.809948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.810017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.810091 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.810170 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.810242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.810312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.810312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.810434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] database.connection = **** {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.810512 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.810660 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.810660 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.810805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.810876 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.810945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.connection = **** {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.811023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.811107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.811179 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.811249 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.811327 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.811399 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.811469 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.811540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.811610 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.811679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.811825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.811900 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.811970 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.812040 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.812111 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.812180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.812261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.812334 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.812334 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.812462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.812554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.812627 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.812706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.812769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.812769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.812954 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.813026 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.813104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.813675 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.813752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.813822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80255) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 11:24:28.813893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.813893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.813893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:24:28.813893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 11:24:28.837709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:28.837965 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:28.838193 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:28.838460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:28.838718 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:28.838943 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:28.846749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:28.847538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:28.874194 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:28.874418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:28.874762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:28.874994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.017347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.017530 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.066548 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.066790 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.066882 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.067026 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.131124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.131316 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.131402 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.131544 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.131638 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.131764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.131845 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.131977 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.132059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.132189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.132270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.132402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.156745 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.156988 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.157132 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.157267 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.157384 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.157484 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.157591 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.157674 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.157817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.157900 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.158038 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.158122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.158253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.158465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.168973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.169237 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.182260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.182375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.182511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.182660 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.182739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.182876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.182957 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.183029 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.183101 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.183235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.183333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.183452 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.183534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80255) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.183667 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:24:29.201535 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.201740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 11:24:29.201847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.202016 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 11:24:29.202099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.202251 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 11:24:29.202331 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.202514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.202616 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 11:24:29.202796 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.202877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.203048 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.203129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.203297 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.203379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.203545 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.214051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.214330 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.214475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.214619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.214778 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.214895 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.215040 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.215313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.223164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.223918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.226405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.226709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.226984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.227216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.227298 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.227493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.227574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.227750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.227839 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.228008 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.228090 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.228270 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.228349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.228529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.228611 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.228790 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.228869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.229044 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.229121 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.229307 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.229573 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.229834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.256395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.257406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.257983 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.258258 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.259043 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.259321 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.263475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.263866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.264145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.264573 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.264957 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.265237 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.271534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.271763 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.272570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.272851 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.276212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.276495 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.276771 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.277045 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.277310 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.277392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.277574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.277665 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.277835 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.277918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.278095 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.278178 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.278392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.278698 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.278966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.279229 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.297113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.297385 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.298833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.299105 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.302619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.303739 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.306196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.306617 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.307378 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.307772 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.311598 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.312819 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.315739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.316977 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.319642 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.319931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.320208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.320484 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.320757 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.320904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.321028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.321201 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.321484 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.321765 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.322038 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.322308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.339957 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.340236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.340497 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.340760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.341533 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.341816 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.342089 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.342354 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.349267 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.350928 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.351015 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.351227 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.352614 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.352898 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.354372 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.354674 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.354949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.355223 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.355504 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.356105 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.356380 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.356655 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.356933 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.357207 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.369295 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.369569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.369851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.370125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.371044 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.371323 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.371608 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.371690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.371888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.371983 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.372207 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.373436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.373723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.374021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.390127 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.392586 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.393479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.395884 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.396891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.398630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.399195 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.400241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.400516 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.400800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.400942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.402583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.402863 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.403150 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.408706 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.410312 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.411018 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.412630 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.413523 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.413807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.414188 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.414461 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.415871 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.416157 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.416543 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.416819 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.424156 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.424439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.424714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.426554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.426852 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.427128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.432518 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80255) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.432795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.433071 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.433342 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.433608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.433975 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.434247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.434548 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.434814 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.434990 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:24:29.435075 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.435273 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.435355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.435552 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.435633 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.435822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.435904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.436088 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.436168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80255) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.436457 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.436733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.437018 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.437287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.437554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.437821 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.438090 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.438460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.438837 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:24:29.475019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e50244f9c0 pid: 80255 (default app) Jun 23 11:24:29.477590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e50244f9c0 pid: 80256 (default app) Jun 23 11:24:29.478433 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaf449-e4d9-4ba7-b2b8-c9da89cb41ea None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:29.478760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaf449-e4d9-4ba7-b2b8-c9da89cb41ea None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:29.479033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaf449-e4d9-4ba7-b2b8-c9da89cb41ea None None] PATH_INFO: `/v3` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:29.483200 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Thu Jun 23 11:24:29 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:38.158302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8b250-1065-4f1b-9eae-2a4e93a8916a None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:38.158952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8b250-1065-4f1b-9eae-2a4e93a8916a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:38.158952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8b250-1065-4f1b-9eae-2a4e93a8916a None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:38.163278 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:24:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:24:38.168719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32df65-a4a8-4c67-9de8-5b945dcee1ea None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:38.169115 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32df65-a4a8-4c67-9de8-5b945dcee1ea None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:38.169273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32df65-a4a8-4c67-9de8-5b945dcee1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:38.241694 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: INFO dbcounter [None req-2a32df65-a4a8-4c67-9de8-5b945dcee1ea None None] Registered counter for database keystone Jun 23 11:24:38.263197 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG oslo_db.sqlalchemy.engines [None req-2a32df65-a4a8-4c67-9de8-5b945dcee1ea None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80255) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 11:24:38.270477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writer thread running {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 11:24:38.452483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG passlib.handlers.bcrypt [None req-2a32df65-a4a8-4c67-9de8-5b945dcee1ea None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80255) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 11:24:38.452821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG passlib.handlers.bcrypt [None req-2a32df65-a4a8-4c67-9de8-5b945dcee1ea None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80255) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 11:24:38.507760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-2a32df65-a4a8-4c67-9de8-5b945dcee1ea None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:24:38.509340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-2a32df65-a4a8-4c67-9de8-5b945dcee1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:38.629380 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:24:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 23 11:24:38.777174 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8b250-1065-4f1b-9eae-2a4e93a8916a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:38.847448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: INFO dbcounter [None req-40c8b250-1065-4f1b-9eae-2a4e93a8916a None None] Registered counter for database keystone Jun 23 11:24:38.867139 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG oslo_db.sqlalchemy.engines [None req-40c8b250-1065-4f1b-9eae-2a4e93a8916a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80256) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 11:24:38.873374 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writer thread running {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 11:24:38.993889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8b250-1065-4f1b-9eae-2a4e93a8916a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:38.998093 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17285ba9-57b2-40c3-96ed-4b98641897d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:38.998787 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17285ba9-57b2-40c3-96ed-4b98641897d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:38.999078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17285ba9-57b2-40c3-96ed-4b98641897d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:38.999360 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17285ba9-57b2-40c3-96ed-4b98641897d1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:39.008346 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17285ba9-57b2-40c3-96ed-4b98641897d1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:39.060216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: WARNING keystone.common.rbac_enforcer.enforcer [None req-17285ba9-57b2-40c3-96ed-4b98641897d1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 11:24:39.060507 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17285ba9-57b2-40c3-96ed-4b98641897d1 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:39.070987 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Thu Jun 23 11:24:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:24:39.076996 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32df65-a4a8-4c67-9de8-5b945dcee1ea None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:39.091238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32df65-a4a8-4c67-9de8-5b945dcee1ea None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:39.096145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d253b3-de94-4fc2-87d1-c7ae38499ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:39.097256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d253b3-de94-4fc2-87d1-c7ae38499ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:39.097539 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d253b3-de94-4fc2-87d1-c7ae38499ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:39.097810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d253b3-de94-4fc2-87d1-c7ae38499ea7 admin admin] PATH_INFO: `/v3/projects/237fd59b9b4946f0a2bdfcf73db9274e` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:39.099315 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d253b3-de94-4fc2-87d1-c7ae38499ea7 admin admin] RBAC: Authorizing `identity:get_project(project_id=237fd59b9b4946f0a2bdfcf73db9274e)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:39.152512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: WARNING keystone.common.rbac_enforcer.enforcer [None req-16d253b3-de94-4fc2-87d1-c7ae38499ea7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 11:24:39.152857 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d253b3-de94-4fc2-87d1-c7ae38499ea7 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:39.157348 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Thu Jun 23 11:24:39 2022] GET /identity/v3/projects/237fd59b9b4946f0a2bdfcf73db9274e => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:41.171321 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd3da8-0623-4330-8948-09567e2e5ec1 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:41.171868 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd3da8-0623-4330-8948-09567e2e5ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:41.171868 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd3da8-0623-4330-8948-09567e2e5ec1 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:41.176534 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:24:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:24:41.181923 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d6d21-08f3-4d8d-ae13-65883ec04ff0 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:41.182338 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d6d21-08f3-4d8d-ae13-65883ec04ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:41.182482 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d6d21-08f3-4d8d-ae13-65883ec04ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:41.202119 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-358d6d21-08f3-4d8d-ae13-65883ec04ff0 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:24:41.203658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-358d6d21-08f3-4d8d-ae13-65883ec04ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:41.211848 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:24:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 23 11:24:41.234315 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd3da8-0623-4330-8948-09567e2e5ec1 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:41.246816 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd3da8-0623-4330-8948-09567e2e5ec1 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:41.250947 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42b1b3-e645-4672-80e8-5238d579b8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:41.252146 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42b1b3-e645-4672-80e8-5238d579b8a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:41.252427 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42b1b3-e645-4672-80e8-5238d579b8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:41.252711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42b1b3-e645-4672-80e8-5238d579b8a2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:41.260348 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42b1b3-e645-4672-80e8-5238d579b8a2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:41.262119 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42b1b3-e645-4672-80e8-5238d579b8a2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:41.270955 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Thu Jun 23 11:24:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 11:24:41.276283 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d6d21-08f3-4d8d-ae13-65883ec04ff0 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:41.285205 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d6d21-08f3-4d8d-ae13-65883ec04ff0 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:41.289460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee6298-2eae-435f-8072-6e547d496e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:41.290318 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee6298-2eae-435f-8072-6e547d496e00 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:41.290615 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee6298-2eae-435f-8072-6e547d496e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:41.290892 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee6298-2eae-435f-8072-6e547d496e00 admin admin] PATH_INFO: `/v3/users/d5118162f0624ca794a0f081b0c2cca5` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:41.292273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee6298-2eae-435f-8072-6e547d496e00 admin admin] RBAC: Authorizing `identity:get_user(user_id=d5118162f0624ca794a0f081b0c2cca5)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:41.294077 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee6298-2eae-435f-8072-6e547d496e00 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:41.295655 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Thu Jun 23 11:24:41 2022] GET /identity/v3/users/d5118162f0624ca794a0f081b0c2cca5 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:43.394330 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c3575-58cb-434a-9cfc-04a23f943e0f None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:43.394900 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c3575-58cb-434a-9cfc-04a23f943e0f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:43.394900 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c3575-58cb-434a-9cfc-04a23f943e0f None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:43.395691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a2ed7-1f33-4ed8-b5f5-00b7b0fb0937 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:43.396050 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a2ed7-1f33-4ed8-b5f5-00b7b0fb0937 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:43.396263 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a2ed7-1f33-4ed8-b5f5-00b7b0fb0937 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:43.396390 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:24:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:24:43.397616 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:24:43.401320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ea1f7-c4b2-4f5d-b198-4341d886c62b None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:43.401620 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ea1f7-c4b2-4f5d-b198-4341d886c62b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:43.401898 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ea1f7-c4b2-4f5d-b198-4341d886c62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:43.402716 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f643a-5edf-4636-accb-358ae1a03596 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:43.403006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f643a-5edf-4636-accb-358ae1a03596 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:43.403283 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f643a-5edf-4636-accb-358ae1a03596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:43.420046 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-699f643a-5edf-4636-accb-358ae1a03596 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:24:43.421211 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-699f643a-5edf-4636-accb-358ae1a03596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:43.442009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG passlib.handlers.bcrypt [None req-cb8ea1f7-c4b2-4f5d-b198-4341d886c62b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80256) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 11:24:43.442423 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG passlib.handlers.bcrypt [None req-cb8ea1f7-c4b2-4f5d-b198-4341d886c62b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80256) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 11:24:43.489623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-cb8ea1f7-c4b2-4f5d-b198-4341d886c62b None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:24:43.491230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-cb8ea1f7-c4b2-4f5d-b198-4341d886c62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:43.504128 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 7/12] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:24:43.526596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f643a-5edf-4636-accb-358ae1a03596 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:43.551086 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:43.595507 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 6/13] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:24:43.618606 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ea1f7-c4b2-4f5d-b198-4341d886c62b None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:43.634857 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f643a-5edf-4636-accb-358ae1a03596 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:43.668954 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05f5b2-c2cb-44bb-8433-3af03e1c0ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:43.669535 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05f5b2-c2cb-44bb-8433-3af03e1c0ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:43.669813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05f5b2-c2cb-44bb-8433-3af03e1c0ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:43.670083 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05f5b2-c2cb-44bb-8433-3af03e1c0ed4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:43.719002 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ea1f7-c4b2-4f5d-b198-4341d886c62b None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:43.721364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:43.739834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae05f5b2-c2cb-44bb-8433-3af03e1c0ed4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:43.741994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae05f5b2-c2cb-44bb-8433-3af03e1c0ed4 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:43.756311 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61452c83-d668-4a3a-96fc-1f833b77a8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:43.757356 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61452c83-d668-4a3a-96fc-1f833b77a8dc None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:43.757638 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61452c83-d668-4a3a-96fc-1f833b77a8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:43.757909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61452c83-d668-4a3a-96fc-1f833b77a8dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:43.765036 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:43.770747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61452c83-d668-4a3a-96fc-1f833b77a8dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:24:43.781870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-61452c83-d668-4a3a-96fc-1f833b77a8dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:24:43.781870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:24:43.781870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:24:43.781870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:24:43.781870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:24:43.781870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:24:43.781870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:24:43.781870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:24:43.781870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:24:43.781870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:24:43.781870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:24:43.781870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:24:43.781870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:24:43.781870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:24:43.781870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:24:43.781870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:24:43.783240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:24:43.783240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:24:43.783240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:24:43.783240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:24:43.783240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:24:43.783240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:24:43.783240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:24:43.783240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:24:43.783240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:24:43.783240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:24:43.783240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:24:43.783240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:24:43.783240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:24:43.783240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:24:43.783240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:24:43.783240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:24:43.784078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:24:43.784078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:24:43.784078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:24:43.784078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:24:43.784078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:24:43.784078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:24:43.784078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:24:43.784078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:24:43.784078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:24:43.784078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:24:43.784078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:24:43.784078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:24:43.784078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:24:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:24:43.789457 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61452c83-d668-4a3a-96fc-1f833b77a8dc None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:43.841226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 8/15] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:24:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 11:24:43.847575 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05f5b2-c2cb-44bb-8433-3af03e1c0ed4 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:43.889003 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61452c83-d668-4a3a-96fc-1f833b77a8dc None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:43.922446 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79660549-a6e8-42af-879a-c5a9f1049d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:43.923695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79660549-a6e8-42af-879a-c5a9f1049d5d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:43.923986 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79660549-a6e8-42af-879a-c5a9f1049d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:43.924263 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79660549-a6e8-42af-879a-c5a9f1049d5d None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:43.925030 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79660549-a6e8-42af-879a-c5a9f1049d5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:24:43.925343 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79660549-a6e8-42af-879a-c5a9f1049d5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:43.926812 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79660549-a6e8-42af-879a-c5a9f1049d5d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:43.938413 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 8/16] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:24:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:24:43.943581 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05f5b2-c2cb-44bb-8433-3af03e1c0ed4 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:43.944109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79660549-a6e8-42af-879a-c5a9f1049d5d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:43.967185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:43.971586 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:43.979557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6eda3e-a5f0-4de0-a918-46c84be6100f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:43.980345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6eda3e-a5f0-4de0-a918-46c84be6100f None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:43.982741 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6eda3e-a5f0-4de0-a918-46c84be6100f None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:43.982741 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6eda3e-a5f0-4de0-a918-46c84be6100f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:43.991017 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6eda3e-a5f0-4de0-a918-46c84be6100f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:43.992501 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6eda3e-a5f0-4de0-a918-46c84be6100f None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:44.013726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-df6eda3e-a5f0-4de0-a918-46c84be6100f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:24:44.013726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:24:44.013726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:24:44.013726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:24:44.013726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:24:44.013726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:24:44.013726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:24:44.013726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:24:44.013726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:24:44.013726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:24:44.013726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:24:44.013726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:24:44.013726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:24:44.013726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:24:44.013726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:24:44.014594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:24:44.014594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:24:44.014594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:24:44.014594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:24:44.014594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:24:44.014594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:24:44.014594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:24:44.014594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:24:44.014594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:24:44.014594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:24:44.014594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:24:44.014594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:24:44.014594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:24:44.014594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:24:44.014594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:24:44.014594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:24:44.014594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:24:44.015409 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:24:44.015409 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:24:44.015409 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:24:44.015409 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:24:44.015409 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 9/17] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:24:44.021457 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6eda3e-a5f0-4de0-a918-46c84be6100f None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:44.046363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79660549-a6e8-42af-879a-c5a9f1049d5d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:44.080525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e3946-68ac-4d5f-bb64-9505b8ebdf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:44.081402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e3946-68ac-4d5f-bb64-9505b8ebdf32 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:44.081689 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e3946-68ac-4d5f-bb64-9505b8ebdf32 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:44.081976 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e3946-68ac-4d5f-bb64-9505b8ebdf32 None admin] PATH_INFO: `/v3/users/d5118162f0624ca794a0f081b0c2cca5` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:44.083534 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e3946-68ac-4d5f-bb64-9505b8ebdf32 None admin] RBAC: Authorizing `identity:get_user(user_id=d5118162f0624ca794a0f081b0c2cca5)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:44.085006 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e3946-68ac-4d5f-bb64-9505b8ebdf32 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:44.086830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 9/18] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:24:43 2022] GET /identity/v3/users/d5118162f0624ca794a0f081b0c2cca5 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:24:44.091890 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e3946-68ac-4d5f-bb64-9505b8ebdf32 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:44.119702 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6eda3e-a5f0-4de0-a918-46c84be6100f None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:44.152821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a7f77-e85f-44cd-9386-f8f1a8c6c86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:44.153482 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a7f77-e85f-44cd-9386-f8f1a8c6c86d None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:44.153777 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a7f77-e85f-44cd-9386-f8f1a8c6c86d None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:44.154057 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a7f77-e85f-44cd-9386-f8f1a8c6c86d None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:44.154817 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a7f77-e85f-44cd-9386-f8f1a8c6c86d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:24:44.155131 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a7f77-e85f-44cd-9386-f8f1a8c6c86d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:44.156524 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a7f77-e85f-44cd-9386-f8f1a8c6c86d None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:44.173997 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 10/19] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:24:44.186624 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:44.194065 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e3946-68ac-4d5f-bb64-9505b8ebdf32 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:44.231266 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e7e1d-baea-449b-9af0-98654d69e4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:44.232077 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e7e1d-baea-449b-9af0-98654d69e4dd None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:44.232360 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e7e1d-baea-449b-9af0-98654d69e4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:44.232633 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e7e1d-baea-449b-9af0-98654d69e4dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:44.233853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e7e1d-baea-449b-9af0-98654d69e4dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:44.235369 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e7e1d-baea-449b-9af0-98654d69e4dd None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:44.237274 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 10/20] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:24:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:24:44.243266 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a7f77-e85f-44cd-9386-f8f1a8c6c86d None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:44.280751 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:44.350422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a7f77-e85f-44cd-9386-f8f1a8c6c86d None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:44.385866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d77f8-84c6-4f59-905d-aec43aa969de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:44.386569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d77f8-84c6-4f59-905d-aec43aa969de None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:44.386846 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d77f8-84c6-4f59-905d-aec43aa969de None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:44.387127 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d77f8-84c6-4f59-905d-aec43aa969de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:44.387959 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d77f8-84c6-4f59-905d-aec43aa969de None admin] RBAC: Adding query filter params (role.id=c9e006de0d6c4812ab29d778e9c8a830, scope.domain.id=default, user.id=d5118162f0624ca794a0f081b0c2cca5) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:24:44.388271 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d77f8-84c6-4f59-905d-aec43aa969de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:44.389724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d77f8-84c6-4f59-905d-aec43aa969de None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:44.397723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 11/21] 10.222.0.13 () {56 vars in 1322 bytes} [Thu Jun 23 11:24:44 2022] GET /identity/v3/role_assignments?user.id=d5118162f0624ca794a0f081b0c2cca5&scope.domain.id=default&role.id=c9e006de0d6c4812ab29d778e9c8a830 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:46.794933 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b8058-5b66-488c-a4d7-ecd3f6d84623 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:46.795411 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b8058-5b66-488c-a4d7-ecd3f6d84623 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:46.795411 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b8058-5b66-488c-a4d7-ecd3f6d84623 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:46.796750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 11/22] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:24:46.806206 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413aef2f-1a8a-4a2c-af0f-b80cad1be986 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:46.811862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413aef2f-1a8a-4a2c-af0f-b80cad1be986 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:46.811862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413aef2f-1a8a-4a2c-af0f-b80cad1be986 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:46.811862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 12/23] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:24:46.816050 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387b839-f459-409f-a32e-9483b96101ba None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:46.816343 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387b839-f459-409f-a32e-9483b96101ba None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:46.816622 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387b839-f459-409f-a32e-9483b96101ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:46.819663 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaccdfd-cb88-4ab6-b6e5-2290ab79c55e None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:46.819949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaccdfd-cb88-4ab6-b6e5-2290ab79c55e None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:46.820229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaccdfd-cb88-4ab6-b6e5-2290ab79c55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:46.866461 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-bcaccdfd-cb88-4ab6-b6e5-2290ab79c55e None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:24:46.867683 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-bcaccdfd-cb88-4ab6-b6e5-2290ab79c55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:46.880906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-1387b839-f459-409f-a32e-9483b96101ba None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:24:46.882101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1387b839-f459-409f-a32e-9483b96101ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:46.984843 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 13/24] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:24:47.031737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaccdfd-cb88-4ab6-b6e5-2290ab79c55e None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:47.035224 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 12/25] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 23 11:24:47.071140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387b839-f459-409f-a32e-9483b96101ba None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:47.080197 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:47.176527 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaccdfd-cb88-4ab6-b6e5-2290ab79c55e None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:47.217985 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4784f37-7b2c-40f6-a305-514f233e7a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:47.218822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4784f37-7b2c-40f6-a305-514f233e7a7a None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:47.219107 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4784f37-7b2c-40f6-a305-514f233e7a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:47.219385 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4784f37-7b2c-40f6-a305-514f233e7a7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:47.238581 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4784f37-7b2c-40f6-a305-514f233e7a7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:47.240290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4784f37-7b2c-40f6-a305-514f233e7a7a None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:47.258376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:47.260670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-f4784f37-7b2c-40f6-a305-514f233e7a7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:24:47.260670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:24:47.260670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:24:47.260670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:24:47.260670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:24:47.260670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:24:47.260670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:24:47.260670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:24:47.260670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:24:47.260670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:24:47.260670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:24:47.260670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:24:47.260670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:24:47.260670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:24:47.260670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:24:47.261628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:24:47.261628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:24:47.261628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:24:47.261628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:24:47.261628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:24:47.261628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:24:47.261628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:24:47.261628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:24:47.261628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:24:47.261628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:24:47.261628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:24:47.261628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:24:47.261628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:24:47.261628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:24:47.261628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:24:47.261628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:24:47.261628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:24:47.262651 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:24:47.262651 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:24:47.262651 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:24:47.262651 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:24:47.264123 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 14/26] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:24:47.265667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3173ab4-aa9a-4467-9329-4a63145595dd None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:47.273856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3173ab4-aa9a-4467-9329-4a63145595dd None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:47.273856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3173ab4-aa9a-4467-9329-4a63145595dd None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:47.273856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 15/27] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:24:47.273856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba14c7-e8a8-4a0a-9576-6b096171f722 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:47.273856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba14c7-e8a8-4a0a-9576-6b096171f722 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:47.273856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba14c7-e8a8-4a0a-9576-6b096171f722 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:47.273856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 16/28] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:24:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:24:47.275017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387b839-f459-409f-a32e-9483b96101ba None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:47.275799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17213cdc-5bf8-449e-b581-e7ef68920e18 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:47.276089 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17213cdc-5bf8-449e-b581-e7ef68920e18 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:47.276366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17213cdc-5bf8-449e-b581-e7ef68920e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:47.293264 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-17213cdc-5bf8-449e-b581-e7ef68920e18 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:24:47.294450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-17213cdc-5bf8-449e-b581-e7ef68920e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:47.315389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982cb5d-c525-49fc-b155-6f4e97a3a4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:47.316396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982cb5d-c525-49fc-b155-6f4e97a3a4b9 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:47.316996 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982cb5d-c525-49fc-b155-6f4e97a3a4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:47.317830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982cb5d-c525-49fc-b155-6f4e97a3a4b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:47.319206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982cb5d-c525-49fc-b155-6f4e97a3a4b9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:47.320847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982cb5d-c525-49fc-b155-6f4e97a3a4b9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:47.369046 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 17/29] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:24:47.370576 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83ec81-614d-443f-bd9d-34e8aaf94d38 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:47.371011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83ec81-614d-443f-bd9d-34e8aaf94d38 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:47.371449 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83ec81-614d-443f-bd9d-34e8aaf94d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:47.392621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-3f83ec81-614d-443f-bd9d-34e8aaf94d38 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:24:47.394095 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-3f83ec81-614d-443f-bd9d-34e8aaf94d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:47.398832 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:47.475880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 18/30] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:24:47.476743 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83ec81-614d-443f-bd9d-34e8aaf94d38 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:47.561073 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 13/31] 10.222.0.13 () {60 vars in 1131 bytes} [Thu Jun 23 11:24:47 2022] POST /identity/v3/roles => generated 219 bytes in 490 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:24:47.567944 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982cb5d-c525-49fc-b155-6f4e97a3a4b9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:47.592223 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83ec81-614d-443f-bd9d-34e8aaf94d38 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:47.646446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda1337-2093-4bfa-8b98-651d670269ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:47.647362 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda1337-2093-4bfa-8b98-651d670269ad None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:47.647786 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda1337-2093-4bfa-8b98-651d670269ad None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:47.648197 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda1337-2093-4bfa-8b98-651d670269ad None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:47.649228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda1337-2093-4bfa-8b98-651d670269ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:24:47.649783 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda1337-2093-4bfa-8b98-651d670269ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:47.651389 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda1337-2093-4bfa-8b98-651d670269ad None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:47.662952 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:47.666459 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 19/32] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:47.668195 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda1337-2093-4bfa-8b98-651d670269ad None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:47.725717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982cb5d-c525-49fc-b155-6f4e97a3a4b9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:47.766514 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a1ce3-fa15-41e4-ab25-15b619e1f28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:47.767706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a1ce3-fa15-41e4-ab25-15b619e1f28e None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:47.768138 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a1ce3-fa15-41e4-ab25-15b619e1f28e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:47.768591 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a1ce3-fa15-41e4-ab25-15b619e1f28e None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:47.769590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a1ce3-fa15-41e4-ab25-15b619e1f28e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:47.771346 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a1ce3-fa15-41e4-ab25-15b619e1f28e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:47.792030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda1337-2093-4bfa-8b98-651d670269ad None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:47.829894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10d16d-4abc-4865-8342-07e0248775d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:47.830517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10d16d-4abc-4865-8342-07e0248775d9 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:47.830517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10d16d-4abc-4865-8342-07e0248775d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:47.830671 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10d16d-4abc-4865-8342-07e0248775d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:47.837055 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 14/33] 10.222.0.13 () {60 vars in 1131 bytes} [Thu Jun 23 11:24:47 2022] POST /identity/v3/roles => generated 225 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:24:47.838489 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f700dc-8981-4bb2-a0b6-4706367786e2 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:47.841412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f700dc-8981-4bb2-a0b6-4706367786e2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:47.841412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f700dc-8981-4bb2-a0b6-4706367786e2 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:47.841412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 15/34] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:24:47.841412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f700dc-8981-4bb2-a0b6-4706367786e2 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:47.854502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10d16d-4abc-4865-8342-07e0248775d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:24:47.862279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:47.869648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-bd10d16d-4abc-4865-8342-07e0248775d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:24:47.869648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:24:47.869648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:24:47.869648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:24:47.869648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:24:47.869648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:24:47.869648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:24:47.869648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:24:47.869648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:24:47.869648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:24:47.869648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:24:47.869648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:24:47.869648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:24:47.869648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:24:47.869648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:24:47.869648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:24:47.870653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:24:47.870653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:24:47.870653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:24:47.870653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:24:47.870653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:24:47.870653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:24:47.870653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:24:47.870653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:24:47.870653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:24:47.870653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:24:47.870653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:24:47.870653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:24:47.870653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:24:47.870653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:24:47.870653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:24:47.870653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:24:47.871439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:24:47.871439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:24:47.871439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:24:47.871439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:24:47.871439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:24:47.871439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:24:47.871439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:24:47.871439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:24:47.871439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:24:47.871439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:24:47.871439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:24:47.871439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:24:47.871439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 20/35] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:24:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:24:47.872703 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ebf20-0c94-4219-b49e-2c64771720c3 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:47.872990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ebf20-0c94-4219-b49e-2c64771720c3 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:47.873264 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ebf20-0c94-4219-b49e-2c64771720c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:47.899637 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-808ebf20-0c94-4219-b49e-2c64771720c3 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:24:47.900778 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-808ebf20-0c94-4219-b49e-2c64771720c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:47.989225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:48.004298 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 21/36] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:24:48.005149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ebf20-0c94-4219-b49e-2c64771720c3 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:48.005876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f700dc-8981-4bb2-a0b6-4706367786e2 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:48.050980 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d58680-13ff-4139-9181-4a04e3098fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:48.052124 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d58680-13ff-4139-9181-4a04e3098fa1 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:48.052635 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d58680-13ff-4139-9181-4a04e3098fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:48.053263 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d58680-13ff-4139-9181-4a04e3098fa1 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:48.054240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d58680-13ff-4139-9181-4a04e3098fa1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:48.056004 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d58680-13ff-4139-9181-4a04e3098fa1 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:48.124909 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ebf20-0c94-4219-b49e-2c64771720c3 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:48.160020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4466bc9-f0e9-4ffd-a5d8-7c294a7de0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:48.160710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4466bc9-f0e9-4ffd-a5d8-7c294a7de0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:48.161015 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4466bc9-f0e9-4ffd-a5d8-7c294a7de0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:48.161307 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4466bc9-f0e9-4ffd-a5d8-7c294a7de0b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:48.162196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4466bc9-f0e9-4ffd-a5d8-7c294a7de0b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:24:48.162511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4466bc9-f0e9-4ffd-a5d8-7c294a7de0b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:48.164020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4466bc9-f0e9-4ffd-a5d8-7c294a7de0b6 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:48.171502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 16/37] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 23 11:24:47 2022] POST /identity/v3/projects => generated 301 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:24:48.171502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d58680-13ff-4139-9181-4a04e3098fa1 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:48.185223 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 22/38] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:24:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:48.191374 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4466bc9-f0e9-4ffd-a5d8-7c294a7de0b6 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:48.255838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:48.258133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:48.310750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d58680-13ff-4139-9181-4a04e3098fa1 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:48.320358 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4466bc9-f0e9-4ffd-a5d8-7c294a7de0b6 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:48.354967 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d558f-a993-41a2-aa60-35175057117a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:48.356168 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d558f-a993-41a2-aa60-35175057117a None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:48.356456 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d558f-a993-41a2-aa60-35175057117a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:48.357050 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d558f-a993-41a2-aa60-35175057117a None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:48.357812 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458d558f-a993-41a2-aa60-35175057117a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:48.359382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458d558f-a993-41a2-aa60-35175057117a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:48.366292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ada923-6960-4c1b-b601-1192e02d55cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:48.366734 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ada923-6960-4c1b-b601-1192e02d55cc None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:48.367384 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ada923-6960-4c1b-b601-1192e02d55cc None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:48.367384 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ada923-6960-4c1b-b601-1192e02d55cc None admin] PATH_INFO: `/v3/users/d5118162f0624ca794a0f081b0c2cca5` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:48.368904 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ada923-6960-4c1b-b601-1192e02d55cc None admin] RBAC: Authorizing `identity:get_user(user_id=d5118162f0624ca794a0f081b0c2cca5)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:48.370313 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ada923-6960-4c1b-b601-1192e02d55cc None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:48.372066 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 23/39] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:24:48 2022] GET /identity/v3/users/d5118162f0624ca794a0f081b0c2cca5 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:24:48.373315 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc129f-4d37-4935-8e71-0eaf4013df9a None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:48.373593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc129f-4d37-4935-8e71-0eaf4013df9a None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:48.373870 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc129f-4d37-4935-8e71-0eaf4013df9a None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:48.375203 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 24/40] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:24:48.384681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc129f-4d37-4935-8e71-0eaf4013df9a None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:48.442931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 17/41] 10.222.0.13 () {60 vars in 1131 bytes} [Thu Jun 23 11:24:48 2022] POST /identity/v3/roles => generated 223 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:24:48.456246 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743b7d7-9251-4a4c-b396-2b9577d08f55 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:48.456537 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743b7d7-9251-4a4c-b396-2b9577d08f55 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:48.456819 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743b7d7-9251-4a4c-b396-2b9577d08f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:48.476230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-0743b7d7-9251-4a4c-b396-2b9577d08f55 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:24:48.477845 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-0743b7d7-9251-4a4c-b396-2b9577d08f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:48.492775 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc129f-4d37-4935-8e71-0eaf4013df9a None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:48.516921 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:48.532246 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9ab21-7747-4cc9-844c-766ab1ef8b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:48.533065 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9ab21-7747-4cc9-844c-766ab1ef8b5f None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:48.533351 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9ab21-7747-4cc9-844c-766ab1ef8b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:48.533634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9ab21-7747-4cc9-844c-766ab1ef8b5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:48.534942 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9ab21-7747-4cc9-844c-766ab1ef8b5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:48.536422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9ab21-7747-4cc9-844c-766ab1ef8b5f None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:48.538312 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 25/42] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:24:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:24:48.543984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9ab21-7747-4cc9-844c-766ab1ef8b5f None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:48.570587 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 18/43] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:24:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:24:48.592889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743b7d7-9251-4a4c-b396-2b9577d08f55 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:48.662047 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:48.663040 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9ab21-7747-4cc9-844c-766ab1ef8b5f None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:48.697361 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8b1d0-c8d6-456d-b2a5-e0091733b5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:48.698160 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8b1d0-c8d6-456d-b2a5-e0091733b5a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:48.698443 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8b1d0-c8d6-456d-b2a5-e0091733b5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:48.698747 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8b1d0-c8d6-456d-b2a5-e0091733b5a3 None admin] PATH_INFO: `/v3/domains/default/users/d5118162f0624ca794a0f081b0c2cca5/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:48.700762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8b1d0-c8d6-456d-b2a5-e0091733b5a3 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d5118162f0624ca794a0f081b0c2cca5, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:48.702223 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8b1d0-c8d6-456d-b2a5-e0091733b5a3 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:48.707593 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743b7d7-9251-4a4c-b396-2b9577d08f55 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:48.741586 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d83ce-0231-4154-a817-6fef01b4d8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:48.742389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d83ce-0231-4154-a817-6fef01b4d8db None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:48.742735 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d83ce-0231-4154-a817-6fef01b4d8db None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:48.743000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d83ce-0231-4154-a817-6fef01b4d8db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:48.744218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d83ce-0231-4154-a817-6fef01b4d8db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:48.745685 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d83ce-0231-4154-a817-6fef01b4d8db None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:48.747344 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 19/44] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:24:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:24:48.753525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d83ce-0231-4154-a817-6fef01b4d8db None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:48.861775 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d83ce-0231-4154-a817-6fef01b4d8db None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:48.886826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:48.899183 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38905f66-c4f7-4bd2-ac72-ace108706a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:48.899965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38905f66-c4f7-4bd2-ac72-ace108706a07 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:48.900255 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38905f66-c4f7-4bd2-ac72-ace108706a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:48.900533 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38905f66-c4f7-4bd2-ac72-ace108706a07 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:48.901251 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38905f66-c4f7-4bd2-ac72-ace108706a07 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:48.902794 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38905f66-c4f7-4bd2-ac72-ace108706a07 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:48.984102 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 26/45] 10.222.0.13 () {58 vars in 1361 bytes} [Thu Jun 23 11:24:48 2022] PUT /identity/v3/domains/default/users/d5118162f0624ca794a0f081b0c2cca5/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:24:48.992096 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 20/46] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 23 11:24:48 2022] POST /identity/v3/projects => generated 312 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:24:51.010218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdb0b2-085e-4388-8cef-793f3abf2431 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:51.010792 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdb0b2-085e-4388-8cef-793f3abf2431 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:51.010875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdb0b2-085e-4388-8cef-793f3abf2431 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:51.015120 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 27/47] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:24:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:24:51.021026 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8000a-58fe-47eb-94b9-04e0813357b1 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:51.021417 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8000a-58fe-47eb-94b9-04e0813357b1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:51.021723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8000a-58fe-47eb-94b9-04e0813357b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:51.029640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5dee71-044e-4f93-9f81-9260a8dcb295 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:51.029933 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5dee71-044e-4f93-9f81-9260a8dcb295 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:51.030219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5dee71-044e-4f93-9f81-9260a8dcb295 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:51.031582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 28/48] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:24:51.037220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00366b8-443d-40e0-a812-3761d3dcd83f None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:51.037514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00366b8-443d-40e0-a812-3761d3dcd83f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:51.037797 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00366b8-443d-40e0-a812-3761d3dcd83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:51.039713 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-e3e8000a-58fe-47eb-94b9-04e0813357b1 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:24:51.040903 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e3e8000a-58fe-47eb-94b9-04e0813357b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:51.054605 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-e00366b8-443d-40e0-a812-3761d3dcd83f None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:24:51.056100 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e00366b8-443d-40e0-a812-3761d3dcd83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:51.100107 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:51.111690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 21/49] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:24:51.122985 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 29/50] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:24:51.135662 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8000a-58fe-47eb-94b9-04e0813357b1 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:51.147191 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00366b8-443d-40e0-a812-3761d3dcd83f None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:51.233771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8000a-58fe-47eb-94b9-04e0813357b1 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:51.236290 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:51.245144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00366b8-443d-40e0-a812-3761d3dcd83f None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:51.271442 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f24f0-9e2a-49ce-8625-fed309c572cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:51.272163 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f24f0-9e2a-49ce-8625-fed309c572cc None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:51.272446 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f24f0-9e2a-49ce-8625-fed309c572cc None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:51.272720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f24f0-9e2a-49ce-8625-fed309c572cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:51.279149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd5e38-d6bc-4055-a8b2-64679cd7b597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:51.279940 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd5e38-d6bc-4055-a8b2-64679cd7b597 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:51.280230 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd5e38-d6bc-4055-a8b2-64679cd7b597 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:51.280510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd5e38-d6bc-4055-a8b2-64679cd7b597 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:51.281691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd5e38-d6bc-4055-a8b2-64679cd7b597 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:51.282984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f24f0-9e2a-49ce-8625-fed309c572cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:24:51.283430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd5e38-d6bc-4055-a8b2-64679cd7b597 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:51.285050 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 30/51] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:24:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:24:51.290525 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd5e38-d6bc-4055-a8b2-64679cd7b597 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:51.290745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-c92f24f0-9e2a-49ce-8625-fed309c572cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:24:51.290745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:24:51.290745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:24:51.290745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:24:51.290745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:24:51.290745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:24:51.290745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:24:51.290745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:24:51.290745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:24:51.290745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:24:51.290745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:24:51.290745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:24:51.290745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:24:51.290745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:24:51.290745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:24:51.290745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:24:51.291575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:24:51.291575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:24:51.291575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:24:51.291575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:24:51.291575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:24:51.291575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:24:51.291575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:24:51.291575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:24:51.291575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:24:51.291575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:24:51.291575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:24:51.291575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:24:51.291575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:24:51.291575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:24:51.291575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:24:51.291575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:24:51.292309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:24:51.292309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:24:51.292309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:24:51.292309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:24:51.292309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:24:51.292309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:24:51.292309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:24:51.292309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:24:51.292309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:24:51.292309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:24:51.292309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:24:51.292309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:24:51.292309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 22/52] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:24:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:24:51.297490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f24f0-9e2a-49ce-8625-fed309c572cc None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:51.338307 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:51.391659 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd5e38-d6bc-4055-a8b2-64679cd7b597 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:51.396160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f24f0-9e2a-49ce-8625-fed309c572cc None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:51.425798 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd3be7-7369-4fb3-98ef-8ac95758be7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:51.426464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd3be7-7369-4fb3-98ef-8ac95758be7b None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:51.426778 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd3be7-7369-4fb3-98ef-8ac95758be7b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:51.427055 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd3be7-7369-4fb3-98ef-8ac95758be7b None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:51.427765 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd3be7-7369-4fb3-98ef-8ac95758be7b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:51.429225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd3be7-7369-4fb3-98ef-8ac95758be7b None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:51.429709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ba3ae-a295-4d7c-947d-1fd556beea6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:51.430368 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ba3ae-a295-4d7c-947d-1fd556beea6e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:51.430676 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ba3ae-a295-4d7c-947d-1fd556beea6e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:51.430952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ba3ae-a295-4d7c-947d-1fd556beea6e None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:51.431700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ba3ae-a295-4d7c-947d-1fd556beea6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:24:51.432008 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ba3ae-a295-4d7c-947d-1fd556beea6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:51.433384 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ba3ae-a295-4d7c-947d-1fd556beea6e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:51.441844 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 23/53] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:24:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:24:51.447497 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ba3ae-a295-4d7c-947d-1fd556beea6e None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:51.474869 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:51.543877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ba3ae-a295-4d7c-947d-1fd556beea6e None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:51.545885 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 31/54] 10.222.0.13 () {60 vars in 1140 bytes} [Thu Jun 23 11:24:51 2022] POST /identity/v3/projects => generated 298 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:24:51.587388 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbafb33-8f30-40c4-aa84-844cff414be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:51.588284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbafb33-8f30-40c4-aa84-844cff414be5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:51.588587 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbafb33-8f30-40c4-aa84-844cff414be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:51.588845 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbafb33-8f30-40c4-aa84-844cff414be5 None admin] PATH_INFO: `/v3/users/d5118162f0624ca794a0f081b0c2cca5` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:51.590135 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbafb33-8f30-40c4-aa84-844cff414be5 None admin] RBAC: Authorizing `identity:get_user(user_id=d5118162f0624ca794a0f081b0c2cca5)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:51.591680 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbafb33-8f30-40c4-aa84-844cff414be5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:51.593428 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 24/55] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:24:51 2022] GET /identity/v3/users/d5118162f0624ca794a0f081b0c2cca5 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:24:51.599928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd3be7-7369-4fb3-98ef-8ac95758be7b None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:51.701598 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:51.705754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd3be7-7369-4fb3-98ef-8ac95758be7b None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:51.740739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cda39-fedb-490b-a945-a9e7ec61137a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:51.741545 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cda39-fedb-490b-a945-a9e7ec61137a None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:51.741837 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cda39-fedb-490b-a945-a9e7ec61137a None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:51.742123 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cda39-fedb-490b-a945-a9e7ec61137a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:51.743344 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6cda39-fedb-490b-a945-a9e7ec61137a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:51.744966 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6cda39-fedb-490b-a945-a9e7ec61137a None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:51.746635 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 32/56] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:24:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:51.752658 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbafb33-8f30-40c4-aa84-844cff414be5 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:51.954782 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:51.961385 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbafb33-8f30-40c4-aa84-844cff414be5 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:51.994932 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e588647-45a0-472a-b122-53dd7bd5436c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:51.995590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e588647-45a0-472a-b122-53dd7bd5436c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:51.995876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e588647-45a0-472a-b122-53dd7bd5436c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:51.996154 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e588647-45a0-472a-b122-53dd7bd5436c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:51.997013 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e588647-45a0-472a-b122-53dd7bd5436c None admin] RBAC: Adding query filter params (role.id=c9e006de0d6c4812ab29d778e9c8a830, scope.domain.id=default, user.id=d5118162f0624ca794a0f081b0c2cca5) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:24:51.997325 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e588647-45a0-472a-b122-53dd7bd5436c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:51.998790 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e588647-45a0-472a-b122-53dd7bd5436c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:52.007926 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 25/57] 10.222.0.13 () {56 vars in 1322 bytes} [Thu Jun 23 11:24:51 2022] GET /identity/v3/role_assignments?user.id=d5118162f0624ca794a0f081b0c2cca5&scope.domain.id=default&role.id=c9e006de0d6c4812ab29d778e9c8a830 => generated 530 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:53.610748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384c0e6-7ae1-46d5-950d-5abc1a7025f8 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:53.611732 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384c0e6-7ae1-46d5-950d-5abc1a7025f8 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:53.611900 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384c0e6-7ae1-46d5-950d-5abc1a7025f8 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:53.613315 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 33/58] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:24:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:24:53.618349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67816f27-37e6-47f8-bf4f-2085bebd7cac None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:53.618806 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67816f27-37e6-47f8-bf4f-2085bebd7cac None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:53.618959 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67816f27-37e6-47f8-bf4f-2085bebd7cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:53.635416 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-67816f27-37e6-47f8-bf4f-2085bebd7cac None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:24:53.636619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-67816f27-37e6-47f8-bf4f-2085bebd7cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:53.701848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 26/59] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:24:53.726897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384c0e6-7ae1-46d5-950d-5abc1a7025f8 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:53.825108 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384c0e6-7ae1-46d5-950d-5abc1a7025f8 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:53.858024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91c872-4283-4f41-ab20-a3a890919fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:53.858818 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91c872-4283-4f41-ab20-a3a890919fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:53.859105 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91c872-4283-4f41-ab20-a3a890919fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:53.859381 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91c872-4283-4f41-ab20-a3a890919fc0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:53.860781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91c872-4283-4f41-ab20-a3a890919fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:53.862223 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91c872-4283-4f41-ab20-a3a890919fc0 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:53.863835 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 34/60] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:24:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:53.869139 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67816f27-37e6-47f8-bf4f-2085bebd7cac None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:53.954963 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:53.965993 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67816f27-37e6-47f8-bf4f-2085bebd7cac None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:53.998732 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b063ee-3451-4452-a922-2cd80ed6b6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:53.999394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b063ee-3451-4452-a922-2cd80ed6b6b5 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:53.999680 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b063ee-3451-4452-a922-2cd80ed6b6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:53.999955 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b063ee-3451-4452-a922-2cd80ed6b6b5 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:54.001226 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b063ee-3451-4452-a922-2cd80ed6b6b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:54.003008 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b063ee-3451-4452-a922-2cd80ed6b6b5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:54.092042 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 27/61] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 23 11:24:53 2022] POST /identity/v3/users => generated 275 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:24:57.269037 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49720887-f882-4aab-99f0-de5aa3180adc None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:57.270050 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49720887-f882-4aab-99f0-de5aa3180adc None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:57.270582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49720887-f882-4aab-99f0-de5aa3180adc None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:57.272177 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 35/62] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:24:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:24:57.277721 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2292fc9-a0ca-4826-9f23-b629689bd926 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:57.278526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2292fc9-a0ca-4826-9f23-b629689bd926 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:57.279032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2292fc9-a0ca-4826-9f23-b629689bd926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:57.299582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a2292fc9-a0ca-4826-9f23-b629689bd926 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:24:57.301079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a2292fc9-a0ca-4826-9f23-b629689bd926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:57.311094 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04194c-f4b4-4e66-b22b-af3ba10aa931 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:57.311672 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04194c-f4b4-4e66-b22b-af3ba10aa931 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:57.312146 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04194c-f4b4-4e66-b22b-af3ba10aa931 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:57.313801 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 36/63] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:24:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:24:57.319395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90187687-fa13-4680-b044-29e1d2300cd3 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:57.319936 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90187687-fa13-4680-b044-29e1d2300cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:57.320409 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90187687-fa13-4680-b044-29e1d2300cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:57.347537 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-90187687-fa13-4680-b044-29e1d2300cd3 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:24:57.351277 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-90187687-fa13-4680-b044-29e1d2300cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:57.410896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:57.439415 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 28/64] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:24:57.465631 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 37/65] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:24:57.467635 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2292fc9-a0ca-4826-9f23-b629689bd926 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:57.493842 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90187687-fa13-4680-b044-29e1d2300cd3 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:57.574942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:57.600693 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2292fc9-a0ca-4826-9f23-b629689bd926 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:57.649205 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94316c-bee6-4bf0-b749-78d15550bc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:57.650846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94316c-bee6-4bf0-b749-78d15550bc9c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:57.651344 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94316c-bee6-4bf0-b749-78d15550bc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:57.652009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94316c-bee6-4bf0-b749-78d15550bc9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:57.660516 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90187687-fa13-4680-b044-29e1d2300cd3 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:57.665693 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94316c-bee6-4bf0-b749-78d15550bc9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:24:57.675584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-4d94316c-bee6-4bf0-b749-78d15550bc9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:24:57.675584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:24:57.675584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:24:57.675584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:24:57.675584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:24:57.675584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:24:57.675584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:24:57.675584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:24:57.675584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:24:57.675584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:24:57.675584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:24:57.675584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:24:57.675584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:24:57.675584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:24:57.675584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:24:57.675584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:24:57.676631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:24:57.676631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:24:57.676631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:24:57.676631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:24:57.676631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:24:57.676631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:24:57.676631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:24:57.676631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:24:57.676631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:24:57.676631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:24:57.676631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:24:57.676631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:24:57.676631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:24:57.676631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:24:57.676631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:24:57.676631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:24:57.677574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:24:57.677574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:24:57.677574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:24:57.677574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:24:57.677574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:24:57.677574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:24:57.677574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:24:57.677574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:24:57.677574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:24:57.677574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:24:57.677574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:24:57.677574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:24:57.685137 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 29/66] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:24:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:24:57.685137 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82274f8-d8ea-4378-a448-3fdae224dac5 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:57.685137 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82274f8-d8ea-4378-a448-3fdae224dac5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:57.685137 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82274f8-d8ea-4378-a448-3fdae224dac5 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:57.685972 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 30/67] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:24:57.686564 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82274f8-d8ea-4378-a448-3fdae224dac5 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:57.709815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c616d-8b76-43fe-80b5-92a684c9ed06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:57.711028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c616d-8b76-43fe-80b5-92a684c9ed06 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:57.711534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c616d-8b76-43fe-80b5-92a684c9ed06 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:57.712021 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c616d-8b76-43fe-80b5-92a684c9ed06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:57.723717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c616d-8b76-43fe-80b5-92a684c9ed06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:24:57.732198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-662c616d-8b76-43fe-80b5-92a684c9ed06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:24:57.732198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:24:57.732198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:24:57.732198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:24:57.732198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:24:57.732198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:24:57.732198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:24:57.732198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:24:57.732198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:24:57.732198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:24:57.732198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:24:57.732198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:24:57.732198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:24:57.732198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:24:57.732198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:24:57.732198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:24:57.733038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:24:57.733038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:24:57.733038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:24:57.733038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:24:57.733038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:24:57.733038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:24:57.733038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:24:57.733038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:24:57.733038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:24:57.733038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:24:57.733038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:24:57.733038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:24:57.733038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:24:57.733038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:24:57.733038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:24:57.733038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:24:57.733789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:24:57.733789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:24:57.733789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:24:57.733789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:24:57.733789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:24:57.733789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:24:57.733789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:24:57.733789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:24:57.733789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:24:57.733789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:24:57.733789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:24:57.733789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:24:57.736981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 38/68] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:24:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:24:57.743225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb278f-a37d-4596-94da-e18093546e1a None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:57.743751 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb278f-a37d-4596-94da-e18093546e1a None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:57.744230 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb278f-a37d-4596-94da-e18093546e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:57.762252 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-d7bb278f-a37d-4596-94da-e18093546e1a None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:24:57.763604 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d7bb278f-a37d-4596-94da-e18093546e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:57.783997 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:57.808571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82274f8-d8ea-4378-a448-3fdae224dac5 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:57.846651 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 39/69] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:24:57.846917 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4e8b3-6e79-4d17-b24d-61b7fb4d25fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:57.847596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4e8b3-6e79-4d17-b24d-61b7fb4d25fe None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:57.847888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4e8b3-6e79-4d17-b24d-61b7fb4d25fe None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:57.848166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4e8b3-6e79-4d17-b24d-61b7fb4d25fe None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:57.848911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4e8b3-6e79-4d17-b24d-61b7fb4d25fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:24:57.849220 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4e8b3-6e79-4d17-b24d-61b7fb4d25fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:57.850639 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4e8b3-6e79-4d17-b24d-61b7fb4d25fe None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:57.851499 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb278f-a37d-4596-94da-e18093546e1a None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:57.852225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:57.863973 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 31/70] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:24:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:57.866235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975eb93f-0f9f-498e-8368-bf94fd2ee628 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:57.866757 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975eb93f-0f9f-498e-8368-bf94fd2ee628 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:57.867865 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975eb93f-0f9f-498e-8368-bf94fd2ee628 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:57.869712 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 32/71] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:24:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:24:57.870843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975eb93f-0f9f-498e-8368-bf94fd2ee628 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:57.960462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb278f-a37d-4596-94da-e18093546e1a None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:57.974861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975eb93f-0f9f-498e-8368-bf94fd2ee628 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:57.994859 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd053197-34f3-41fd-b277-44ea21898faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:57.995525 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd053197-34f3-41fd-b277-44ea21898faa None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:57.995808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd053197-34f3-41fd-b277-44ea21898faa None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:57.996085 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd053197-34f3-41fd-b277-44ea21898faa None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:57.996827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd053197-34f3-41fd-b277-44ea21898faa None admin] RBAC: Adding query filter params (name=member) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:24:57.997132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd053197-34f3-41fd-b277-44ea21898faa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:57.998553 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd053197-34f3-41fd-b277-44ea21898faa None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:58.007233 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 40/72] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:24:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:58.008057 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd053197-34f3-41fd-b277-44ea21898faa None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:58.008590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae62bab-baa6-43fd-8ce3-7eb746408011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:58.009521 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae62bab-baa6-43fd-8ce3-7eb746408011 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:58.009834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae62bab-baa6-43fd-8ce3-7eb746408011 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:58.010114 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae62bab-baa6-43fd-8ce3-7eb746408011 None admin] PATH_INFO: `/v3/users/d5118162f0624ca794a0f081b0c2cca5` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:58.011348 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae62bab-baa6-43fd-8ce3-7eb746408011 None admin] RBAC: Authorizing `identity:get_user(user_id=d5118162f0624ca794a0f081b0c2cca5)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:58.012804 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae62bab-baa6-43fd-8ce3-7eb746408011 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:58.014431 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 33/73] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:24:57 2022] GET /identity/v3/users/d5118162f0624ca794a0f081b0c2cca5 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:58.019839 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bb7a8-0921-484e-8ce9-5c8eb160f52e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:58.020135 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bb7a8-0921-484e-8ce9-5c8eb160f52e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:58.020420 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bb7a8-0921-484e-8ce9-5c8eb160f52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:58.028745 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:58.036862 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-fc1bb7a8-0921-484e-8ce9-5c8eb160f52e None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:24:58.038068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-fc1bb7a8-0921-484e-8ce9-5c8eb160f52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:58.097751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:58.105761 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 34/74] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:24:58.107065 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7d153-5702-418b-ba92-dcba5af6ad16 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:58.107350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7d153-5702-418b-ba92-dcba5af6ad16 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:58.107941 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7d153-5702-418b-ba92-dcba5af6ad16 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:58.109053 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 35/75] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:24:58.109600 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7d153-5702-418b-ba92-dcba5af6ad16 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:58.111625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd053197-34f3-41fd-b277-44ea21898faa None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:58.147953 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e54d9-23c0-465a-b4a1-dda062bb2e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:58.148768 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e54d9-23c0-465a-b4a1-dda062bb2e20 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:58.149055 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e54d9-23c0-465a-b4a1-dda062bb2e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:58.149337 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e54d9-23c0-465a-b4a1-dda062bb2e20 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:58.160265 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e54d9-23c0-465a-b4a1-dda062bb2e20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:24:58.168276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-655e54d9-23c0-465a-b4a1-dda062bb2e20 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:24:58.168276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:24:58.168276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:24:58.168276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:24:58.168276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:24:58.168276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:24:58.168276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:24:58.168276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:24:58.168276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:24:58.168276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:24:58.168276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:24:58.168276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:24:58.168276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:24:58.168276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:24:58.168276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:24:58.168276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:24:58.169145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:24:58.169145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:24:58.169145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:24:58.169145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:24:58.169145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:24:58.169145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:24:58.169145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:24:58.169145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:24:58.169145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:24:58.169145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:24:58.169145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:24:58.169145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:24:58.169145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:24:58.169145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:24:58.169145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:24:58.169145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:24:58.169908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:24:58.169908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:24:58.169908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:24:58.169908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:24:58.169908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:24:58.169908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:24:58.169908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:24:58.169908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:24:58.169908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:24:58.169908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:24:58.169908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:24:58.169908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:24:58.169908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 41/76] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:24:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:24:58.175396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e54d9-23c0-465a-b4a1-dda062bb2e20 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:58.214079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7d153-5702-418b-ba92-dcba5af6ad16 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:58.248408 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b8157-13de-4654-be93-bb3174b3e64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:58.249283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b8157-13de-4654-be93-bb3174b3e64d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:58.249577 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b8157-13de-4654-be93-bb3174b3e64d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:58.249859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b8157-13de-4654-be93-bb3174b3e64d None admin] PATH_INFO: `/v3/users/7e575b3126124c04a504aee5827ae3d3` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:58.251639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:58.264966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b8157-13de-4654-be93-bb3174b3e64d None admin] RBAC: Authorizing `identity:get_user(user_id=7e575b3126124c04a504aee5827ae3d3)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:58.266994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b8157-13de-4654-be93-bb3174b3e64d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:58.268767 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 36/77] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:24:58 2022] GET /identity/v3/users/7e575b3126124c04a504aee5827ae3d3 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:58.269940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4159c-587c-4a3c-9750-f384b3304ce1 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:58.270230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4159c-587c-4a3c-9750-f384b3304ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:58.270511 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4159c-587c-4a3c-9750-f384b3304ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:58.279026 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e54d9-23c0-465a-b4a1-dda062bb2e20 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:58.287860 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c8a4159c-587c-4a3c-9750-f384b3304ce1 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:24:58.289057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c8a4159c-587c-4a3c-9750-f384b3304ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:58.313282 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82863a80-2da4-4efe-a4f4-f12fd879f03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:58.313936 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82863a80-2da4-4efe-a4f4-f12fd879f03a None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:58.314221 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82863a80-2da4-4efe-a4f4-f12fd879f03a None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:58.314510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82863a80-2da4-4efe-a4f4-f12fd879f03a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:58.335978 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:58.356594 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 37/78] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:24:58.357597 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4159c-587c-4a3c-9750-f384b3304ce1 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:58.388118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82863a80-2da4-4efe-a4f4-f12fd879f03a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:58.389613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82863a80-2da4-4efe-a4f4-f12fd879f03a None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:58.458023 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4159c-587c-4a3c-9750-f384b3304ce1 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:58.465322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:58.489090 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 42/79] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:24:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:24:58.490103 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82863a80-2da4-4efe-a4f4-f12fd879f03a None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:58.495113 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebab72-c785-4fa1-bbfb-ae69762382aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:58.495924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebab72-c785-4fa1-bbfb-ae69762382aa None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:58.496237 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebab72-c785-4fa1-bbfb-ae69762382aa None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:58.496686 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebab72-c785-4fa1-bbfb-ae69762382aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:58.507230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebab72-c785-4fa1-bbfb-ae69762382aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:24:58.514908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-53ebab72-c785-4fa1-bbfb-ae69762382aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:24:58.514908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:24:58.514908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:24:58.514908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:24:58.514908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:24:58.514908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:24:58.514908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:24:58.514908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:24:58.514908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:24:58.514908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:24:58.514908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:24:58.514908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:24:58.514908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:24:58.514908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:24:58.514908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:24:58.514908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:24:58.516502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:24:58.516502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:24:58.516502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:24:58.516502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:24:58.516502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:24:58.516502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:24:58.516502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:24:58.516502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:24:58.516502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:24:58.516502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:24:58.516502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:24:58.516502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:24:58.516502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:24:58.516502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:24:58.516502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:24:58.516502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:24:58.517349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:24:58.517349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:24:58.517349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:24:58.517349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:24:58.517349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:24:58.517349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:24:58.517349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:24:58.517349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:24:58.517349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:24:58.517349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:24:58.517349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:24:58.517349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:24:58.517349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 38/80] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:24:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:24:58.517995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebab72-c785-4fa1-bbfb-ae69762382aa None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:58.564830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:58.593252 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82863a80-2da4-4efe-a4f4-f12fd879f03a None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:58.622333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebab72-c785-4fa1-bbfb-ae69762382aa None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:58.627823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f661c-2e00-46ed-9ca2-3c48e94b11aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:58.628493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f661c-2e00-46ed-9ca2-3c48e94b11aa None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:58.628781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f661c-2e00-46ed-9ca2-3c48e94b11aa None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:58.629060 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f661c-2e00-46ed-9ca2-3c48e94b11aa None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:58.630219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f661c-2e00-46ed-9ca2-3c48e94b11aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:24:58.630548 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f661c-2e00-46ed-9ca2-3c48e94b11aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:58.631941 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f661c-2e00-46ed-9ca2-3c48e94b11aa None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:58.639918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 43/81] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:24:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:58.640506 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f661c-2e00-46ed-9ca2-3c48e94b11aa None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:58.656060 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b90fcf-fc5b-4e2f-a64e-caacfb9273f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:58.657512 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b90fcf-fc5b-4e2f-a64e-caacfb9273f4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:58.658004 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b90fcf-fc5b-4e2f-a64e-caacfb9273f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:58.658293 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b90fcf-fc5b-4e2f-a64e-caacfb9273f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:58.679419 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:58.725745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b90fcf-fc5b-4e2f-a64e-caacfb9273f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:58.727289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b90fcf-fc5b-4e2f-a64e-caacfb9273f4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:58.738280 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f661c-2e00-46ed-9ca2-3c48e94b11aa None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:58.769077 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:58.770969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202515a6-5b30-4a95-b90c-eeea6d2babd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:58.771812 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202515a6-5b30-4a95-b90c-eeea6d2babd8 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:58.772107 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202515a6-5b30-4a95-b90c-eeea6d2babd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:58.772382 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202515a6-5b30-4a95-b90c-eeea6d2babd8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:58.773574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202515a6-5b30-4a95-b90c-eeea6d2babd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:58.775063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202515a6-5b30-4a95-b90c-eeea6d2babd8 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:58.776673 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 44/82] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:24:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:58.777426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202515a6-5b30-4a95-b90c-eeea6d2babd8 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:58.827373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 39/83] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:24:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:24:58.828337 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b90fcf-fc5b-4e2f-a64e-caacfb9273f4 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:58.879246 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202515a6-5b30-4a95-b90c-eeea6d2babd8 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:58.886820 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:58.912949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236df374-ed61-4dc1-8229-81545bde3929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:58.913797 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236df374-ed61-4dc1-8229-81545bde3929 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:58.914077 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236df374-ed61-4dc1-8229-81545bde3929 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:58.914399 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236df374-ed61-4dc1-8229-81545bde3929 None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:58.915951 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236df374-ed61-4dc1-8229-81545bde3929 None admin] RBAC: Authorizing `identity:get_project(project_id=d20d807ab0054a0d994c8b287d262cba)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:58.917813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236df374-ed61-4dc1-8229-81545bde3929 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:58.919559 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 45/84] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:24:58 2022] GET /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:58.920321 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236df374-ed61-4dc1-8229-81545bde3929 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:58.931251 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b90fcf-fc5b-4e2f-a64e-caacfb9273f4 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:58.965300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86925ec7-4a25-417c-83b4-71e716c3091d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:58.965972 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86925ec7-4a25-417c-83b4-71e716c3091d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:58.966255 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86925ec7-4a25-417c-83b4-71e716c3091d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:58.966554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86925ec7-4a25-417c-83b4-71e716c3091d None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:58.967312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86925ec7-4a25-417c-83b4-71e716c3091d None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:24:58.967621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86925ec7-4a25-417c-83b4-71e716c3091d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:58.969008 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86925ec7-4a25-417c-83b4-71e716c3091d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:58.977829 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 40/85] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:24:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:58.978404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86925ec7-4a25-417c-83b4-71e716c3091d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:58.981827 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:59.021858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236df374-ed61-4dc1-8229-81545bde3929 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:59.054485 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26521670-69a2-48d9-9bb6-04cf188c69de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:59.055368 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26521670-69a2-48d9-9bb6-04cf188c69de None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:59.055650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26521670-69a2-48d9-9bb6-04cf188c69de None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:59.055926 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26521670-69a2-48d9-9bb6-04cf188c69de None admin] PATH_INFO: `/v3/users/7e575b3126124c04a504aee5827ae3d3` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:59.057107 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26521670-69a2-48d9-9bb6-04cf188c69de None admin] RBAC: Authorizing `identity:get_user(user_id=7e575b3126124c04a504aee5827ae3d3)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:59.058557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26521670-69a2-48d9-9bb6-04cf188c69de None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:59.060234 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 46/86] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:24:58 2022] GET /identity/v3/users/7e575b3126124c04a504aee5827ae3d3 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:59.060979 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26521670-69a2-48d9-9bb6-04cf188c69de None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:59.081086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86925ec7-4a25-417c-83b4-71e716c3091d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:59.103756 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:59.113773 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113943a8-4c2a-4efa-90fd-7cc41f3f561f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:59.114508 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113943a8-4c2a-4efa-90fd-7cc41f3f561f None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:59.114824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113943a8-4c2a-4efa-90fd-7cc41f3f561f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:59.115102 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113943a8-4c2a-4efa-90fd-7cc41f3f561f None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:59.115820 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113943a8-4c2a-4efa-90fd-7cc41f3f561f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:59.117281 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113943a8-4c2a-4efa-90fd-7cc41f3f561f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:59.162310 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26521670-69a2-48d9-9bb6-04cf188c69de None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:59.195451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 41/87] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 23 11:24:58 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:24:59.196326 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113943a8-4c2a-4efa-90fd-7cc41f3f561f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:59.200340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98de3d-3072-484c-a41d-4f07b55339e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:59.201237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98de3d-3072-484c-a41d-4f07b55339e2 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:59.201524 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98de3d-3072-484c-a41d-4f07b55339e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:59.201802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98de3d-3072-484c-a41d-4f07b55339e2 None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:59.203123 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98de3d-3072-484c-a41d-4f07b55339e2 None admin] RBAC: Authorizing `identity:get_project(project_id=d20d807ab0054a0d994c8b287d262cba)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:59.204705 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98de3d-3072-484c-a41d-4f07b55339e2 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:59.206971 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 47/88] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:24:59 2022] GET /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:59.207543 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98de3d-3072-484c-a41d-4f07b55339e2 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:59.273137 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:59.323058 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113943a8-4c2a-4efa-90fd-7cc41f3f561f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:59.324067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98de3d-3072-484c-a41d-4f07b55339e2 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:59.337470 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:59.362093 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291a261-a6b5-435d-9475-b35488c1d5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:59.363387 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291a261-a6b5-435d-9475-b35488c1d5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:59.363387 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291a261-a6b5-435d-9475-b35488c1d5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:59.363387 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291a261-a6b5-435d-9475-b35488c1d5f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:59.364223 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291a261-a6b5-435d-9475-b35488c1d5f8 None admin] RBAC: Adding query filter params (role.id=c9e006de0d6c4812ab29d778e9c8a830, scope.project.id=d20d807ab0054a0d994c8b287d262cba, user.id=d5118162f0624ca794a0f081b0c2cca5) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:24:59.364558 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291a261-a6b5-435d-9475-b35488c1d5f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:59.366008 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291a261-a6b5-435d-9475-b35488c1d5f8 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:59.370708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f89d5-0592-48c6-9332-4644f2499341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:59.371588 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f89d5-0592-48c6-9332-4644f2499341 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:59.371873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f89d5-0592-48c6-9332-4644f2499341 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:59.372152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f89d5-0592-48c6-9332-4644f2499341 None admin] PATH_INFO: `/v3/users/7e575b3126124c04a504aee5827ae3d3` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:59.373373 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f89d5-0592-48c6-9332-4644f2499341 None admin] RBAC: Authorizing `identity:get_user(user_id=7e575b3126124c04a504aee5827ae3d3)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:59.374870 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f89d5-0592-48c6-9332-4644f2499341 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:59.375276 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 42/89] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:24:59 2022] GET /identity/v3/role_assignments?user.id=d5118162f0624ca794a0f081b0c2cca5&scope.project.id=d20d807ab0054a0d994c8b287d262cba&role.id=c9e006de0d6c4812ab29d778e9c8a830 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:59.376641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291a261-a6b5-435d-9475-b35488c1d5f8 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:59.377209 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 48/90] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:24:59 2022] GET /identity/v3/users/7e575b3126124c04a504aee5827ae3d3 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:59.377974 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f89d5-0592-48c6-9332-4644f2499341 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:59.515620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f89d5-0592-48c6-9332-4644f2499341 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:59.525686 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291a261-a6b5-435d-9475-b35488c1d5f8 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:59.563748 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:59.570281 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d06f85-620f-4003-b1a7-cf9522f4ac19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:59.570980 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d06f85-620f-4003-b1a7-cf9522f4ac19 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:59.571271 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d06f85-620f-4003-b1a7-cf9522f4ac19 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:59.571550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d06f85-620f-4003-b1a7-cf9522f4ac19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:59.572474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d06f85-620f-4003-b1a7-cf9522f4ac19 None admin] RBAC: Adding query filter params (role.id=75f08cd0671f436c9f968976aa8baa7f, scope.project.id=d20d807ab0054a0d994c8b287d262cba, user.id=7e575b3126124c04a504aee5827ae3d3) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:24:59.580636 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d06f85-620f-4003-b1a7-cf9522f4ac19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:59.580636 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d06f85-620f-4003-b1a7-cf9522f4ac19 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:59.583095 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 49/91] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:24:59 2022] GET /identity/v3/role_assignments?user.id=7e575b3126124c04a504aee5827ae3d3&scope.project.id=d20d807ab0054a0d994c8b287d262cba&role.id=75f08cd0671f436c9f968976aa8baa7f => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:24:59.583940 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d06f85-620f-4003-b1a7-cf9522f4ac19 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:59.584046 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf1caf-65f2-475e-aa0d-697850add623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:59.584606 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf1caf-65f2-475e-aa0d-697850add623 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:59.584895 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf1caf-65f2-475e-aa0d-697850add623 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:59.585184 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf1caf-65f2-475e-aa0d-697850add623 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:59.655294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:59.696137 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf1caf-65f2-475e-aa0d-697850add623 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:59.699292 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf1caf-65f2-475e-aa0d-697850add623 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:59.773504 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d06f85-620f-4003-b1a7-cf9522f4ac19 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:59.841110 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d681871-3afa-4c0d-9b2a-d8e32a82a932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:59.842069 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d681871-3afa-4c0d-9b2a-d8e32a82a932 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:59.842500 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d681871-3afa-4c0d-9b2a-d8e32a82a932 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:59.842963 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d681871-3afa-4c0d-9b2a-d8e32a82a932 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:59.885522 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 43/92] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:24:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:24:59.891937 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf1caf-65f2-475e-aa0d-697850add623 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:59.920278 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:59.991019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d681871-3afa-4c0d-9b2a-d8e32a82a932 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:59.992747 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d681871-3afa-4c0d-9b2a-d8e32a82a932 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:00.041057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf1caf-65f2-475e-aa0d-697850add623 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:00.070925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:00.083681 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b983e11-d61e-4725-a929-0f906785540d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:00.085218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b983e11-d61e-4725-a929-0f906785540d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:00.085644 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b983e11-d61e-4725-a929-0f906785540d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:00.086082 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b983e11-d61e-4725-a929-0f906785540d None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:00.087763 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b983e11-d61e-4725-a929-0f906785540d None admin] RBAC: Authorizing `identity:get_project(project_id=d20d807ab0054a0d994c8b287d262cba)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:00.089519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b983e11-d61e-4725-a929-0f906785540d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:00.091765 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 44/93] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:24:59 2022] GET /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:00.102012 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b983e11-d61e-4725-a929-0f906785540d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:00.165432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 50/94] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:24:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:00.174890 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d681871-3afa-4c0d-9b2a-d8e32a82a932 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:00.294174 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:00.310933 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b983e11-d61e-4725-a929-0f906785540d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:00.361764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d721d49-1fb4-4866-b608-c41efa56f7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:00.362522 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d721d49-1fb4-4866-b608-c41efa56f7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:00.362765 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d721d49-1fb4-4866-b608-c41efa56f7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:00.363049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d721d49-1fb4-4866-b608-c41efa56f7f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:00.363897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d721d49-1fb4-4866-b608-c41efa56f7f4 None admin] RBAC: Adding query filter params (role.id=25e99b2a08914aa0ab18cc4955f25364, scope.project.id=d20d807ab0054a0d994c8b287d262cba, user.id=7e575b3126124c04a504aee5827ae3d3) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:00.364210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d721d49-1fb4-4866-b608-c41efa56f7f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:00.366795 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d721d49-1fb4-4866-b608-c41efa56f7f4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:00.384258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 45/95] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:25:00 2022] GET /identity/v3/role_assignments?user.id=7e575b3126124c04a504aee5827ae3d3&scope.project.id=d20d807ab0054a0d994c8b287d262cba&role.id=25e99b2a08914aa0ab18cc4955f25364 => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:00.406623 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d681871-3afa-4c0d-9b2a-d8e32a82a932 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:00.456584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8f108-0fad-4fd3-994a-faa0af00731d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:00.457490 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8f108-0fad-4fd3-994a-faa0af00731d None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:00.457781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8f108-0fad-4fd3-994a-faa0af00731d None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:00.458062 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8f108-0fad-4fd3-994a-faa0af00731d None admin] PATH_INFO: `/v3/projects/740b5771a1034d5f986d75ce345e365b` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:00.459384 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8f108-0fad-4fd3-994a-faa0af00731d None admin] RBAC: Authorizing `identity:get_project(project_id=740b5771a1034d5f986d75ce345e365b)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:00.460853 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8f108-0fad-4fd3-994a-faa0af00731d None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:00.462569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 51/96] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:00 2022] GET /identity/v3/projects/740b5771a1034d5f986d75ce345e365b => generated 312 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:00.468703 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d721d49-1fb4-4866-b608-c41efa56f7f4 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:00.605063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d721d49-1fb4-4866-b608-c41efa56f7f4 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:00.645162 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a5be23-04c0-4b0c-b103-08fdcea9ed6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:00.646206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5be23-04c0-4b0c-b103-08fdcea9ed6c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:00.646717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5be23-04c0-4b0c-b103-08fdcea9ed6c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:00.647213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5be23-04c0-4b0c-b103-08fdcea9ed6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:00.648243 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5be23-04c0-4b0c-b103-08fdcea9ed6c None admin] RBAC: Adding query filter params (role.id=75f08cd0671f436c9f968976aa8baa7f, scope.project.id=740b5771a1034d5f986d75ce345e365b, user.id=7e575b3126124c04a504aee5827ae3d3) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:00.648750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5be23-04c0-4b0c-b103-08fdcea9ed6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:00.650407 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5be23-04c0-4b0c-b103-08fdcea9ed6c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:00.657633 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 46/97] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:25:00 2022] GET /identity/v3/role_assignments?user.id=7e575b3126124c04a504aee5827ae3d3&scope.project.id=740b5771a1034d5f986d75ce345e365b&role.id=75f08cd0671f436c9f968976aa8baa7f => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:01.867461 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829168ef-501f-45e3-9145-dc7a6065c7c2 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:01.867963 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829168ef-501f-45e3-9145-dc7a6065c7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:01.868067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829168ef-501f-45e3-9145-dc7a6065c7c2 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:01.869384 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 52/98] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:01.874697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44866ed8-ebb2-48e9-8d18-727b1a628617 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:01.875528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44866ed8-ebb2-48e9-8d18-727b1a628617 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:01.876008 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44866ed8-ebb2-48e9-8d18-727b1a628617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:01.903647 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-44866ed8-ebb2-48e9-8d18-727b1a628617 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:01.905172 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-44866ed8-ebb2-48e9-8d18-727b1a628617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:01.921380 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:01.956451 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d0af5-f746-47e2-81cf-e86779d84ff9 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:01.957069 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d0af5-f746-47e2-81cf-e86779d84ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:01.957545 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d0af5-f746-47e2-81cf-e86779d84ff9 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:01.959418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 53/99] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:01.964068 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290eee1a-8b16-48cf-ba1e-e647dcd432f2 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:01.964592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290eee1a-8b16-48cf-ba1e-e647dcd432f2 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:01.965068 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290eee1a-8b16-48cf-ba1e-e647dcd432f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:01.995831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-290eee1a-8b16-48cf-ba1e-e647dcd432f2 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:01.997382 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-290eee1a-8b16-48cf-ba1e-e647dcd432f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:02.008156 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:02.010195 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 47/100] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:02.056170 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44866ed8-ebb2-48e9-8d18-727b1a628617 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:02.116082 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 54/101] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:02.151179 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290eee1a-8b16-48cf-ba1e-e647dcd432f2 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:02.213950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44866ed8-ebb2-48e9-8d18-727b1a628617 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:02.275257 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a43ad-c933-4a5d-bdfc-da2b44deb02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:02.276411 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a43ad-c933-4a5d-bdfc-da2b44deb02c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:02.276838 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a43ad-c933-4a5d-bdfc-da2b44deb02c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:02.277375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a43ad-c933-4a5d-bdfc-da2b44deb02c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:02.291548 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a43ad-c933-4a5d-bdfc-da2b44deb02c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:02.308738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-122a43ad-c933-4a5d-bdfc-da2b44deb02c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:25:02.308738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:02.308738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:02.308738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:02.308738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:02.308738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:02.308738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:02.308738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:02.308738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:02.308738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:02.308738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:02.308738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:02.308738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:02.308738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:02.308738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:02.308738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:02.309629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:02.309629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:02.309629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:02.309629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:02.309629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:02.309629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:02.309629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:02.309629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:02.309629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:02.309629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:02.309629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:02.309629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:02.309629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:02.309629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:02.309629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:02.309629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:02.310379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:02.310379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:02.310379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:02.310379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:02.310379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:02.310379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:02.310379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:02.310379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:02.310379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:02.310379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:02.310379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:25:02.310379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:02.313401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 48/102] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:25:02.322650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290eee1a-8b16-48cf-ba1e-e647dcd432f2 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:02.337460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a43ad-c933-4a5d-bdfc-da2b44deb02c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:02.363882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:02.389978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393937cf-8c65-42eb-a299-0697fee4e513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:02.392403 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393937cf-8c65-42eb-a299-0697fee4e513 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:02.392872 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393937cf-8c65-42eb-a299-0697fee4e513 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:02.393346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393937cf-8c65-42eb-a299-0697fee4e513 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:02.409643 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:02.412771 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393937cf-8c65-42eb-a299-0697fee4e513 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:02.428902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-393937cf-8c65-42eb-a299-0697fee4e513 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:25:02.428902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:02.428902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:02.428902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:02.428902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:02.428902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:02.428902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:02.428902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:02.428902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:02.428902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:02.428902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:02.428902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:02.428902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:02.428902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:02.428902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:02.428902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:02.429816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:02.429816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:02.429816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:02.429816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:02.429816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:02.429816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:02.429816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:02.429816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:02.429816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:02.429816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:02.429816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:02.429816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:02.429816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:02.429816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:02.429816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:02.429816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:02.430609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:02.430609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:02.430609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:02.430609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:02.430609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:02.430609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:02.430609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:02.430609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:02.430609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:02.430609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:02.430609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:25:02.430609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:02.432996 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 55/103] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:25:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:02.439693 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393937cf-8c65-42eb-a299-0697fee4e513 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:02.477057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a43ad-c933-4a5d-bdfc-da2b44deb02c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:02.513811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001255de-913b-4088-8464-bf14876a3768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:02.514725 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001255de-913b-4088-8464-bf14876a3768 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:02.515117 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001255de-913b-4088-8464-bf14876a3768 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:02.515500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001255de-913b-4088-8464-bf14876a3768 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:02.516407 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001255de-913b-4088-8464-bf14876a3768 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:02.516986 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001255de-913b-4088-8464-bf14876a3768 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:02.518401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001255de-913b-4088-8464-bf14876a3768 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:02.527411 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 49/104] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:02.533972 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001255de-913b-4088-8464-bf14876a3768 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:02.611664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:02.643729 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001255de-913b-4088-8464-bf14876a3768 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:02.662444 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393937cf-8c65-42eb-a299-0697fee4e513 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:02.710115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4cd17-69ec-4ce9-9766-30eba2b83c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:02.711885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4cd17-69ec-4ce9-9766-30eba2b83c9e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:02.712172 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4cd17-69ec-4ce9-9766-30eba2b83c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:02.712451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4cd17-69ec-4ce9-9766-30eba2b83c9e None admin] PATH_INFO: `/v3/users/785770e376794f45a62cee26e45bee8f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:02.733349 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9861c5-01ff-48cc-96da-b0c407b57b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:02.734033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9861c5-01ff-48cc-96da-b0c407b57b58 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:02.734328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9861c5-01ff-48cc-96da-b0c407b57b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:02.735156 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9861c5-01ff-48cc-96da-b0c407b57b58 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:02.736009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9861c5-01ff-48cc-96da-b0c407b57b58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:02.736321 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9861c5-01ff-48cc-96da-b0c407b57b58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:02.737728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9861c5-01ff-48cc-96da-b0c407b57b58 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:02.742561 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c4cd17-69ec-4ce9-9766-30eba2b83c9e None admin] RBAC: Authorizing `identity:get_user(user_id=785770e376794f45a62cee26e45bee8f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:02.744027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c4cd17-69ec-4ce9-9766-30eba2b83c9e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:02.751711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 50/105] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:02 2022] GET /identity/v3/users/785770e376794f45a62cee26e45bee8f => generated 289 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:02.755943 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4cd17-69ec-4ce9-9766-30eba2b83c9e None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:02.762815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 56/106] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:02.769250 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9861c5-01ff-48cc-96da-b0c407b57b58 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:02.874231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4cd17-69ec-4ce9-9766-30eba2b83c9e None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:02.907076 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:02.913518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116a2fc-b30f-49e5-bb84-82d0dc797aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:02.914354 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116a2fc-b30f-49e5-bb84-82d0dc797aed None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:02.914807 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116a2fc-b30f-49e5-bb84-82d0dc797aed None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:02.915150 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116a2fc-b30f-49e5-bb84-82d0dc797aed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:02.932200 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:02.992282 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116a2fc-b30f-49e5-bb84-82d0dc797aed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:02.993935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116a2fc-b30f-49e5-bb84-82d0dc797aed None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:03.020780 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9861c5-01ff-48cc-96da-b0c407b57b58 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:03.093688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7d7a0-447f-4bbc-a1b8-26e526fd55c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:03.095469 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7d7a0-447f-4bbc-a1b8-26e526fd55c6 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:03.095692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7d7a0-447f-4bbc-a1b8-26e526fd55c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:03.095984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7d7a0-447f-4bbc-a1b8-26e526fd55c6 None admin] PATH_INFO: `/v3/users/d5118162f0624ca794a0f081b0c2cca5` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:03.098117 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7d7a0-447f-4bbc-a1b8-26e526fd55c6 None admin] RBAC: Authorizing `identity:get_user(user_id=d5118162f0624ca794a0f081b0c2cca5)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:03.099685 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7d7a0-447f-4bbc-a1b8-26e526fd55c6 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:03.101458 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 57/107] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:02 2022] GET /identity/v3/users/d5118162f0624ca794a0f081b0c2cca5 => generated 247 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:03.103267 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b2b6e-e969-4c91-bb43-1276fbf05e46 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:03.103558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b2b6e-e969-4c91-bb43-1276fbf05e46 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:03.103907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b2b6e-e969-4c91-bb43-1276fbf05e46 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:03.106098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 58/108] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:03.107532 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d16877-23a8-4c04-9057-507b6e88c90f None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:03.107828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d16877-23a8-4c04-9057-507b6e88c90f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:03.108302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d16877-23a8-4c04-9057-507b6e88c90f None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:03.110756 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 59/109] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:03.111698 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d16877-23a8-4c04-9057-507b6e88c90f None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:03.129150 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 51/110] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:03.135178 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35eb15-2f5d-44d8-b125-0ff0bd12c9af None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:03.135555 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35eb15-2f5d-44d8-b125-0ff0bd12c9af None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:03.135918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35eb15-2f5d-44d8-b125-0ff0bd12c9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:03.153293 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3c35eb15-2f5d-44d8-b125-0ff0bd12c9af None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:03.154836 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3c35eb15-2f5d-44d8-b125-0ff0bd12c9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:03.187378 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:03.228115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 52/111] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:03.229643 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98cd5b-2089-4f24-a952-09fb4fa718ef None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:03.230042 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98cd5b-2089-4f24-a952-09fb4fa718ef None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:03.230428 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98cd5b-2089-4f24-a952-09fb4fa718ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:03.249613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d16877-23a8-4c04-9057-507b6e88c90f None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:03.257298 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-9c98cd5b-2089-4f24-a952-09fb4fa718ef None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:03.258525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9c98cd5b-2089-4f24-a952-09fb4fa718ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:03.288138 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:03.296876 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07125bf8-7737-45cd-a5d0-547ca8eb84f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:03.298004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07125bf8-7737-45cd-a5d0-547ca8eb84f9 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:03.298433 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07125bf8-7737-45cd-a5d0-547ca8eb84f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:03.299182 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07125bf8-7737-45cd-a5d0-547ca8eb84f9 None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:03.301576 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07125bf8-7737-45cd-a5d0-547ca8eb84f9 None admin] RBAC: Authorizing `identity:get_project(project_id=d20d807ab0054a0d994c8b287d262cba)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:03.303219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07125bf8-7737-45cd-a5d0-547ca8eb84f9 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:03.305101 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 60/112] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:03 2022] GET /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:03.306012 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07125bf8-7737-45cd-a5d0-547ca8eb84f9 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:03.346958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 53/113] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:03.348071 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98cd5b-2089-4f24-a952-09fb4fa718ef None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:03.440027 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07125bf8-7737-45cd-a5d0-547ca8eb84f9 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:03.469344 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:03.476040 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b6e050-33c9-43a6-bbef-ad1ec5b90270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:03.476994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6e050-33c9-43a6-bbef-ad1ec5b90270 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:03.477279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6e050-33c9-43a6-bbef-ad1ec5b90270 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:03.477559 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6e050-33c9-43a6-bbef-ad1ec5b90270 None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:03.479002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b6e050-33c9-43a6-bbef-ad1ec5b90270 None admin] RBAC: Authorizing `identity:get_project(project_id=d20d807ab0054a0d994c8b287d262cba)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:03.480337 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b6e050-33c9-43a6-bbef-ad1ec5b90270 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:03.481959 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98cd5b-2089-4f24-a952-09fb4fa718ef None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:03.492339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 61/114] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:03 2022] GET /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:03.493186 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b6e050-33c9-43a6-bbef-ad1ec5b90270 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:03.526665 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101b56e-701c-45c5-be4a-c65f6d55c0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:03.527957 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101b56e-701c-45c5-be4a-c65f6d55c0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:03.528615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101b56e-701c-45c5-be4a-c65f6d55c0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:03.529449 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101b56e-701c-45c5-be4a-c65f6d55c0e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:03.531083 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d101b56e-701c-45c5-be4a-c65f6d55c0e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:03.534074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d101b56e-701c-45c5-be4a-c65f6d55c0e0 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:03.535915 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 54/115] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:03.536842 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101b56e-701c-45c5-be4a-c65f6d55c0e0 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:03.573991 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:03.612169 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b6e050-33c9-43a6-bbef-ad1ec5b90270 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:03.650464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea1a98-c459-4452-a581-d24ed37cc051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:03.651390 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea1a98-c459-4452-a581-d24ed37cc051 None admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:03.651681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea1a98-c459-4452-a581-d24ed37cc051 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:03.651961 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea1a98-c459-4452-a581-d24ed37cc051 None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba/users/d5118162f0624ca794a0f081b0c2cca5/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:03.654023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea1a98-c459-4452-a581-d24ed37cc051 None admin] RBAC: Authorizing `identity:create_grant(project_id=d20d807ab0054a0d994c8b287d262cba, user_id=d5118162f0624ca794a0f081b0c2cca5, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:03.655510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea1a98-c459-4452-a581-d24ed37cc051 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:03.668687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101b56e-701c-45c5-be4a-c65f6d55c0e0 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:03.703276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 62/116] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:25:03 2022] PUT /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba/users/d5118162f0624ca794a0f081b0c2cca5/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:25:03.704215 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfbb4a-47c8-4dad-8de8-5454eb141d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:03.704699 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea1a98-c459-4452-a581-d24ed37cc051 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:03.705027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfbb4a-47c8-4dad-8de8-5454eb141d58 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:03.705315 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfbb4a-47c8-4dad-8de8-5454eb141d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:03.705597 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfbb4a-47c8-4dad-8de8-5454eb141d58 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:03.718577 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfbb4a-47c8-4dad-8de8-5454eb141d58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:03.727155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-87cfbb4a-47c8-4dad-8de8-5454eb141d58 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:03.727155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:03.727155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:03.727155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:03.727155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:03.727155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:03.727155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:03.727155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:03.727155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:03.727155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:03.727155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:03.727155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:03.727155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:03.727155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:03.727155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:03.727155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:03.728079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:03.728079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:03.728079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:03.728079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:03.728079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:03.728079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:03.728079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:03.728079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:03.728079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:03.728079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:03.728079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:03.728079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:03.728079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:03.728079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:03.728079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:03.728079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:03.734231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:03.734231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:03.734231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:03.734231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:03.734231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:03.734231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:03.734231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:03.734231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:03.734231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:03.734231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:03.734231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:03.734231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:03.734231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 55/117] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:03.734231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfbb4a-47c8-4dad-8de8-5454eb141d58 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:03.756991 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:03.828612 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea1a98-c459-4452-a581-d24ed37cc051 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:03.852465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfbb4a-47c8-4dad-8de8-5454eb141d58 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:03.864182 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:03.874395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531ab81-2b34-46d0-a397-d60883fec32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:03.875686 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531ab81-2b34-46d0-a397-d60883fec32b None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:03.875979 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531ab81-2b34-46d0-a397-d60883fec32b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:03.876261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531ab81-2b34-46d0-a397-d60883fec32b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:03.877253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b531ab81-2b34-46d0-a397-d60883fec32b None admin] RBAC: Adding query filter params (role.id=c8e4f6d521b04ad39661b74575a8b9d3, scope.project.id=d20d807ab0054a0d994c8b287d262cba, user.id=785770e376794f45a62cee26e45bee8f) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:03.877569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b531ab81-2b34-46d0-a397-d60883fec32b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:03.879079 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b531ab81-2b34-46d0-a397-d60883fec32b None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:03.884995 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 63/118] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:25:03 2022] GET /identity/v3/role_assignments?user.id=785770e376794f45a62cee26e45bee8f&scope.project.id=d20d807ab0054a0d994c8b287d262cba&role.id=c8e4f6d521b04ad39661b74575a8b9d3 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:03.892913 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0b929-2a73-46e0-8ab7-7ee9bdb5e97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:03.893600 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0b929-2a73-46e0-8ab7-7ee9bdb5e97f None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:03.893704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8ac57-64a9-43ad-b03e-e7c5629c1b28 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:03.893906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0b929-2a73-46e0-8ab7-7ee9bdb5e97f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:03.894166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0b929-2a73-46e0-8ab7-7ee9bdb5e97f None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:03.894599 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8ac57-64a9-43ad-b03e-e7c5629c1b28 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:03.895034 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0b929-2a73-46e0-8ab7-7ee9bdb5e97f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:03.895269 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8ac57-64a9-43ad-b03e-e7c5629c1b28 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:03.896563 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0b929-2a73-46e0-8ab7-7ee9bdb5e97f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:03.897163 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 64/119] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:03.898873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275c4f6-d4ff-4b44-ae3a-12fc557da8b7 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:03.899356 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275c4f6-d4ff-4b44-ae3a-12fc557da8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:03.899825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275c4f6-d4ff-4b44-ae3a-12fc557da8b7 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:03.902481 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 65/120] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:03.903565 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275c4f6-d4ff-4b44-ae3a-12fc557da8b7 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:03.981058 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 56/121] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 23 11:25:03 2022] POST /identity/v3/projects => generated 302 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:25:03.983476 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44210bbd-d571-4347-8560-81b5d930c9cf None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:03.983921 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44210bbd-d571-4347-8560-81b5d930c9cf None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:03.985440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44210bbd-d571-4347-8560-81b5d930c9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:04.006297 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-44210bbd-d571-4347-8560-81b5d930c9cf None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:04.008917 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-44210bbd-d571-4347-8560-81b5d930c9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:04.083230 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275c4f6-d4ff-4b44-ae3a-12fc557da8b7 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:04.099752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:04.104985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 57/122] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:04.107753 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6972a-8199-46fc-9959-c61a884f508d None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:04.108385 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6972a-8199-46fc-9959-c61a884f508d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:04.108864 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6972a-8199-46fc-9959-c61a884f508d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:04.130717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-2bb6972a-8199-46fc-9959-c61a884f508d None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:04.132566 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2bb6972a-8199-46fc-9959-c61a884f508d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:04.153661 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edd8d1-8899-4ac1-ba57-0e5aae02b1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:04.154836 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edd8d1-8899-4ac1-ba57-0e5aae02b1ba None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:04.154946 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edd8d1-8899-4ac1-ba57-0e5aae02b1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:04.155215 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edd8d1-8899-4ac1-ba57-0e5aae02b1ba None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:04.156043 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edd8d1-8899-4ac1-ba57-0e5aae02b1ba None admin] RBAC: Adding query filter params (name=member) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:04.157021 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edd8d1-8899-4ac1-ba57-0e5aae02b1ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:04.158656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edd8d1-8899-4ac1-ba57-0e5aae02b1ba None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:04.171034 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 66/123] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:04.171883 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edd8d1-8899-4ac1-ba57-0e5aae02b1ba None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:04.232120 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 58/124] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:04.233408 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6972a-8199-46fc-9959-c61a884f508d None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:04.247755 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:04.307432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edd8d1-8899-4ac1-ba57-0e5aae02b1ba None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:04.351166 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c5086-e1af-4f95-80b5-693c5f159e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:04.352049 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c5086-e1af-4f95-80b5-693c5f159e2d None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:04.352340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c5086-e1af-4f95-80b5-693c5f159e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:04.352619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c5086-e1af-4f95-80b5-693c5f159e2d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:04.366885 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c5086-e1af-4f95-80b5-693c5f159e2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:04.376769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-492c5086-e1af-4f95-80b5-693c5f159e2d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:04.376769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:04.376769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:04.376769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:04.376769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:04.376769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:04.376769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:04.376769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:04.376769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:04.376769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:04.376769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:04.376769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:04.376769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:04.376769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:04.376769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:04.376769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:04.377645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:04.377645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:04.377645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:04.377645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:04.377645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:04.377645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:04.377645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:04.377645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:04.377645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:04.377645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:04.377645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:04.377645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:04.377645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:04.377645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:04.377645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:04.377645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:04.378395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:04.378395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:04.378395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:04.378395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:04.378395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:04.378395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:04.378395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:04.378395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:04.378395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:04.378395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:04.378395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:04.378395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:04.378395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 67/125] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:25:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:04.379088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c5086-e1af-4f95-80b5-693c5f159e2d None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:04.400836 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6972a-8199-46fc-9959-c61a884f508d None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:04.434861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:04.441425 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b82d7-dc33-4673-ac40-1701156be4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:04.442310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b82d7-dc33-4673-ac40-1701156be4fa None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:04.442710 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b82d7-dc33-4673-ac40-1701156be4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:04.442994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b82d7-dc33-4673-ac40-1701156be4fa None admin] PATH_INFO: `/v3/users/7e575b3126124c04a504aee5827ae3d3` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:04.444668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b82d7-dc33-4673-ac40-1701156be4fa None admin] RBAC: Authorizing `identity:get_user(user_id=7e575b3126124c04a504aee5827ae3d3)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:04.446216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b82d7-dc33-4673-ac40-1701156be4fa None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:04.448198 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 59/126] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:04 2022] GET /identity/v3/users/7e575b3126124c04a504aee5827ae3d3 => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:04.449120 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b82d7-dc33-4673-ac40-1701156be4fa None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:04.586475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c5086-e1af-4f95-80b5-693c5f159e2d None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:04.600164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b82d7-dc33-4673-ac40-1701156be4fa None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:04.606884 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:04.645225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4b07b-97f2-4f23-a747-16a7dc87c2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:04.646024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4b07b-97f2-4f23-a747-16a7dc87c2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:04.646317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4b07b-97f2-4f23-a747-16a7dc87c2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:04.646624 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4b07b-97f2-4f23-a747-16a7dc87c2e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:04.663789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4b07b-97f2-4f23-a747-16a7dc87c2e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:04.670112 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77096b2f-3bcd-4317-b1db-c2fbf50c46da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:04.670817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77096b2f-3bcd-4317-b1db-c2fbf50c46da None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:04.671108 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77096b2f-3bcd-4317-b1db-c2fbf50c46da None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:04.671390 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77096b2f-3bcd-4317-b1db-c2fbf50c46da None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:04.672197 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77096b2f-3bcd-4317-b1db-c2fbf50c46da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:04.672514 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77096b2f-3bcd-4317-b1db-c2fbf50c46da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:04.673911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77096b2f-3bcd-4317-b1db-c2fbf50c46da None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:04.680218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-fbb4b07b-97f2-4f23-a747-16a7dc87c2e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:04.680218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:04.680218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:04.680218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:04.680218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:04.680218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:04.680218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:04.680218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:04.680218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:04.680218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:04.680218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:04.680218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:04.680218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:04.680218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:04.680218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:04.680218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:04.681132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:04.681132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:04.681132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:04.681132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:04.681132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:04.681132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:04.681132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:04.681132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:04.681132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:04.681132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:04.681132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:04.681132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:04.681132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:04.681132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:04.681132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:04.681132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:04.681928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:04.681928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:04.681928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:04.681928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:04.681928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:04.681928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:04.681928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:04.681928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:04.681928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:04.681928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:04.681928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:04.681928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:04.682568 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 68/127] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:04.686082 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4b07b-97f2-4f23-a747-16a7dc87c2e8 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:04.691746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 60/128] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:25:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:04.697427 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77096b2f-3bcd-4317-b1db-c2fbf50c46da None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:04.803149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4b07b-97f2-4f23-a747-16a7dc87c2e8 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:04.845269 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401500d7-ec8e-44ec-844e-236f26a74e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:04.846463 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401500d7-ec8e-44ec-844e-236f26a74e5d None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:04.846976 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401500d7-ec8e-44ec-844e-236f26a74e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:04.847452 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:04.849201 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401500d7-ec8e-44ec-844e-236f26a74e5d None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:04.850811 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401500d7-ec8e-44ec-844e-236f26a74e5d None admin] RBAC: Authorizing `identity:get_project(project_id=d20d807ab0054a0d994c8b287d262cba)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:04.852558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401500d7-ec8e-44ec-844e-236f26a74e5d None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:04.854463 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 69/129] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:04 2022] GET /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:04.855654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401500d7-ec8e-44ec-844e-236f26a74e5d None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:04.920860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:04.940758 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77096b2f-3bcd-4317-b1db-c2fbf50c46da None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:04.995364 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401500d7-ec8e-44ec-844e-236f26a74e5d None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:05.035254 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db852745-4e7e-4be1-86c1-6dc3ad456945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:05.036229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db852745-4e7e-4be1-86c1-6dc3ad456945 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:05.036727 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db852745-4e7e-4be1-86c1-6dc3ad456945 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:05.037107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee23ae-3d10-4dcf-b7e4-91841340ff19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:05.037739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db852745-4e7e-4be1-86c1-6dc3ad456945 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:05.038124 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee23ae-3d10-4dcf-b7e4-91841340ff19 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:05.038482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee23ae-3d10-4dcf-b7e4-91841340ff19 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:05.038855 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee23ae-3d10-4dcf-b7e4-91841340ff19 None admin] PATH_INFO: `/v3/users/7e575b3126124c04a504aee5827ae3d3` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:05.039321 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db852745-4e7e-4be1-86c1-6dc3ad456945 None admin] RBAC: Adding query filter params (name=member) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:05.039848 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db852745-4e7e-4be1-86c1-6dc3ad456945 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:05.040140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee23ae-3d10-4dcf-b7e4-91841340ff19 None admin] RBAC: Authorizing `identity:get_user(user_id=7e575b3126124c04a504aee5827ae3d3)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:05.041622 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee23ae-3d10-4dcf-b7e4-91841340ff19 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:05.041902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db852745-4e7e-4be1-86c1-6dc3ad456945 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:05.055735 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 61/131] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:04 2022] GET /identity/v3/users/7e575b3126124c04a504aee5827ae3d3 => generated 275 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:05.056024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 70/131] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:05.057114 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee23ae-3d10-4dcf-b7e4-91841340ff19 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:05.058509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db852745-4e7e-4be1-86c1-6dc3ad456945 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:05.173112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db852745-4e7e-4be1-86c1-6dc3ad456945 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:05.175417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:05.212497 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6fc1f-21cd-4ac2-8611-1ed3fd95ae12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:05.214036 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6fc1f-21cd-4ac2-8611-1ed3fd95ae12 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:05.214618 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6fc1f-21cd-4ac2-8611-1ed3fd95ae12 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:05.215124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6fc1f-21cd-4ac2-8611-1ed3fd95ae12 None admin] PATH_INFO: `/v3/users/7e575b3126124c04a504aee5827ae3d3` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:05.217285 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6fc1f-21cd-4ac2-8611-1ed3fd95ae12 None admin] RBAC: Authorizing `identity:get_user(user_id=7e575b3126124c04a504aee5827ae3d3)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:05.219745 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6fc1f-21cd-4ac2-8611-1ed3fd95ae12 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:05.226838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 71/132] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:05 2022] GET /identity/v3/users/7e575b3126124c04a504aee5827ae3d3 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:05.227985 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6fc1f-21cd-4ac2-8611-1ed3fd95ae12 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:05.344098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6fc1f-21cd-4ac2-8611-1ed3fd95ae12 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:05.349586 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee23ae-3d10-4dcf-b7e4-91841340ff19 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:05.389686 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fccc4-e930-4d0d-b113-121ae4e89b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:05.391676 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fccc4-e930-4d0d-b113-121ae4e89b4a None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:05.392359 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fccc4-e930-4d0d-b113-121ae4e89b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:05.392938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fccc4-e930-4d0d-b113-121ae4e89b4a None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:05.394390 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77fccc4-e930-4d0d-b113-121ae4e89b4a None admin] RBAC: Authorizing `identity:get_project(project_id=d20d807ab0054a0d994c8b287d262cba)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:05.396233 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77fccc4-e930-4d0d-b113-121ae4e89b4a None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:05.398351 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 72/133] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:05 2022] GET /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:05.399575 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fccc4-e930-4d0d-b113-121ae4e89b4a None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:05.402881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:05.426406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018148bb-65e1-4959-9c0d-a68820743f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:05.427405 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018148bb-65e1-4959-9c0d-a68820743f18 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:05.427696 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018148bb-65e1-4959-9c0d-a68820743f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:05.428047 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018148bb-65e1-4959-9c0d-a68820743f18 None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba/users/7e575b3126124c04a504aee5827ae3d3/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:05.435441 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018148bb-65e1-4959-9c0d-a68820743f18 None admin] RBAC: Authorizing `identity:create_grant(project_id=d20d807ab0054a0d994c8b287d262cba, user_id=7e575b3126124c04a504aee5827ae3d3, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:05.436950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018148bb-65e1-4959-9c0d-a68820743f18 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:05.447341 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:05.514060 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fccc4-e930-4d0d-b113-121ae4e89b4a None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:05.529288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 62/134] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:25:05 2022] PUT /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba/users/7e575b3126124c04a504aee5827ae3d3/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:25:05.530697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018148bb-65e1-4959-9c0d-a68820743f18 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:05.589843 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc7155-7709-46d9-b6d5-43355f6451ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:05.591108 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc7155-7709-46d9-b6d5-43355f6451ec None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:05.591810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc7155-7709-46d9-b6d5-43355f6451ec None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:05.592627 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc7155-7709-46d9-b6d5-43355f6451ec None admin] PATH_INFO: `/v3/projects/740b5771a1034d5f986d75ce345e365b` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:05.598207 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc7155-7709-46d9-b6d5-43355f6451ec None admin] RBAC: Authorizing `identity:get_project(project_id=740b5771a1034d5f986d75ce345e365b)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:05.599983 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc7155-7709-46d9-b6d5-43355f6451ec None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:05.601957 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 73/135] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:05 2022] GET /identity/v3/projects/740b5771a1034d5f986d75ce345e365b => generated 312 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:05.608900 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc7155-7709-46d9-b6d5-43355f6451ec None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:05.732130 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018148bb-65e1-4959-9c0d-a68820743f18 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:05.790479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc7155-7709-46d9-b6d5-43355f6451ec None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:05.803121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:05.829948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49559e4e-d73d-4360-aac5-1fd855b66edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:05.830897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49559e4e-d73d-4360-aac5-1fd855b66edf None admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:05.831188 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49559e4e-d73d-4360-aac5-1fd855b66edf None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:05.831469 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49559e4e-d73d-4360-aac5-1fd855b66edf None admin] PATH_INFO: `/v3/projects/740b5771a1034d5f986d75ce345e365b/users/7e575b3126124c04a504aee5827ae3d3/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:05.834073 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49559e4e-d73d-4360-aac5-1fd855b66edf None admin] RBAC: Authorizing `identity:create_grant(project_id=740b5771a1034d5f986d75ce345e365b, user_id=7e575b3126124c04a504aee5827ae3d3, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:05.835807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49559e4e-d73d-4360-aac5-1fd855b66edf None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:05.840181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145d390-63b8-4ac8-8a7a-c0842b1377ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:05.840181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145d390-63b8-4ac8-8a7a-c0842b1377ae None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:05.840469 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145d390-63b8-4ac8-8a7a-c0842b1377ae None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:05.840755 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145d390-63b8-4ac8-8a7a-c0842b1377ae None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba/users/7e575b3126124c04a504aee5827ae3d3/roles/25e99b2a08914aa0ab18cc4955f25364` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:05.844862 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145d390-63b8-4ac8-8a7a-c0842b1377ae None admin] RBAC: Authorizing `identity:create_grant(project_id=d20d807ab0054a0d994c8b287d262cba, user_id=7e575b3126124c04a504aee5827ae3d3, role_id=25e99b2a08914aa0ab18cc4955f25364)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:05.847336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145d390-63b8-4ac8-8a7a-c0842b1377ae None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:05.922554 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 74/136] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:25:05 2022] PUT /identity/v3/projects/740b5771a1034d5f986d75ce345e365b/users/7e575b3126124c04a504aee5827ae3d3/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:25:05.957117 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 63/137] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:25:05 2022] PUT /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba/users/7e575b3126124c04a504aee5827ae3d3/roles/25e99b2a08914aa0ab18cc4955f25364 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:25:06.064415 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d312375-9ff7-47d9-9a91-f9f28c108fce None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:06.064751 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d312375-9ff7-47d9-9a91-f9f28c108fce None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:06.064988 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d312375-9ff7-47d9-9a91-f9f28c108fce None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:06.066490 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 75/138] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:06.072687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33024686-81b0-4482-bcc4-e8ecc2519914 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:06.073183 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33024686-81b0-4482-bcc4-e8ecc2519914 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:06.073250 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33024686-81b0-4482-bcc4-e8ecc2519914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:06.096819 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-33024686-81b0-4482-bcc4-e8ecc2519914 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:06.098042 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-33024686-81b0-4482-bcc4-e8ecc2519914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:06.159366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:06.189239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 64/139] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:06.237774 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d312375-9ff7-47d9-9a91-f9f28c108fce None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:06.355896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d312375-9ff7-47d9-9a91-f9f28c108fce None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:06.397092 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb3bd9-b61c-464f-beb7-b33c6d3fdcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:06.397924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb3bd9-b61c-464f-beb7-b33c6d3fdcef None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:06.398208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb3bd9-b61c-464f-beb7-b33c6d3fdcef None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:06.398487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb3bd9-b61c-464f-beb7-b33c6d3fdcef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:06.415909 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb3bd9-b61c-464f-beb7-b33c6d3fdcef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:06.424761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-7ceb3bd9-b61c-464f-beb7-b33c6d3fdcef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:25:06.424761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:06.424761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:06.424761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:06.424761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:06.424761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:06.424761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:06.424761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:06.424761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:06.424761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:06.424761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:06.424761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:06.424761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:06.424761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:06.424761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:06.424761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:06.425654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:06.425654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:06.425654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:06.425654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:06.425654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:06.425654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:06.425654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:06.425654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:06.425654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:06.425654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:06.425654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:06.425654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:06.425654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:06.425654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:06.425654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:06.425654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:06.426466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:06.426466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:06.426466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:06.426466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:06.426466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:06.426466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:06.426466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:06.426466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:06.426466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:06.426466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:06.426466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:25:06.426466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:06.426466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 76/140] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:25:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:06.433414 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33024686-81b0-4482-bcc4-e8ecc2519914 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:06.565546 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33024686-81b0-4482-bcc4-e8ecc2519914 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:06.592189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2db5d7-64d9-4b3e-a8ac-8244f349263c None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:06.592503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2db5d7-64d9-4b3e-a8ac-8244f349263c None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:06.592759 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2db5d7-64d9-4b3e-a8ac-8244f349263c None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:06.594918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 77/141] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:25:06.600283 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a170379-a259-4e01-9909-2fd40f2d13d5 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:06.600582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a170379-a259-4e01-9909-2fd40f2d13d5 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:06.600866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a170379-a259-4e01-9909-2fd40f2d13d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:06.610897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc15bfa8-b48b-4626-aaa8-b62b7990eaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:06.611569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15bfa8-b48b-4626-aaa8-b62b7990eaac None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:06.611853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15bfa8-b48b-4626-aaa8-b62b7990eaac None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:06.612129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15bfa8-b48b-4626-aaa8-b62b7990eaac None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:06.612875 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc15bfa8-b48b-4626-aaa8-b62b7990eaac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:06.613181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc15bfa8-b48b-4626-aaa8-b62b7990eaac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:06.614574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc15bfa8-b48b-4626-aaa8-b62b7990eaac None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:06.627829 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 65/142] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:06.633017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc15bfa8-b48b-4626-aaa8-b62b7990eaac None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:06.641652 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-9a170379-a259-4e01-9909-2fd40f2d13d5 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:06.643088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-9a170379-a259-4e01-9909-2fd40f2d13d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:06.679004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:06.706450 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:06.757451 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 78/143] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:06.802510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a170379-a259-4e01-9909-2fd40f2d13d5 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:06.821595 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc15bfa8-b48b-4626-aaa8-b62b7990eaac None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:06.892041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66a39c-6281-458d-b4a1-166ee5541e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:06.893008 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66a39c-6281-458d-b4a1-166ee5541e74 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:06.893299 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66a39c-6281-458d-b4a1-166ee5541e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:06.893629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66a39c-6281-458d-b4a1-166ee5541e74 None admin] PATH_INFO: `/v3/users/d5118162f0624ca794a0f081b0c2cca5` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:06.895823 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf66a39c-6281-458d-b4a1-166ee5541e74 None admin] RBAC: Authorizing `identity:get_user(user_id=d5118162f0624ca794a0f081b0c2cca5)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:06.897374 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf66a39c-6281-458d-b4a1-166ee5541e74 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:06.899204 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 66/144] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:06 2022] GET /identity/v3/users/d5118162f0624ca794a0f081b0c2cca5 => generated 247 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:06.900218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac3cf1-b261-4a24-9d32-391246240e6c None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:06.900507 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac3cf1-b261-4a24-9d32-391246240e6c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:06.900788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac3cf1-b261-4a24-9d32-391246240e6c None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:06.908516 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 67/145] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:06 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:06.911831 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac3cf1-b261-4a24-9d32-391246240e6c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:06.971459 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a170379-a259-4e01-9909-2fd40f2d13d5 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:07.034470 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4748eea-85c5-4e83-b116-a3a0cd766638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:07.036558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4748eea-85c5-4e83-b116-a3a0cd766638 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:07.037204 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4748eea-85c5-4e83-b116-a3a0cd766638 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:07.038864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4748eea-85c5-4e83-b116-a3a0cd766638 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:07.059553 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4748eea-85c5-4e83-b116-a3a0cd766638 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:07.070304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-c4748eea-85c5-4e83-b116-a3a0cd766638 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:25:07.070304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:07.070304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:07.070304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:07.070304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:07.070304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:07.070304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:07.070304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:07.070304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:07.070304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:07.070304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:07.070304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:07.070304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:07.070304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:07.070304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:07.070304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:07.071236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:07.071236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:07.071236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:07.071236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:07.071236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:07.071236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:07.071236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:07.071236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:07.071236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:07.071236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:07.071236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:07.071236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:07.071236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:07.071236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:07.071236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:07.071236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:07.071990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:07.071990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:07.071990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:07.071990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:07.071990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:07.071990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:07.071990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:07.071990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:07.071990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:07.071990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:07.071990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:25:07.071990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:07.074343 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 79/146] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:07.080538 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cb916-b43e-42ec-bb57-421ba8836cde None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:07.081102 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cb916-b43e-42ec-bb57-421ba8836cde None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:07.081620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cb916-b43e-42ec-bb57-421ba8836cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:07.101128 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c72cb916-b43e-42ec-bb57-421ba8836cde None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:07.102667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c72cb916-b43e-42ec-bb57-421ba8836cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:07.114744 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:07.120156 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac3cf1-b261-4a24-9d32-391246240e6c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:07.135795 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:07.180630 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578565d-ada8-4b29-b4a6-a0844738602b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:07.181809 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578565d-ada8-4b29-b4a6-a0844738602b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:07.182565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578565d-ada8-4b29-b4a6-a0844738602b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:07.183371 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578565d-ada8-4b29-b4a6-a0844738602b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:07.236085 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 80/147] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:07.236780 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cb916-b43e-42ec-bb57-421ba8836cde None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:07.289806 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578565d-ada8-4b29-b4a6-a0844738602b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:07.291647 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578565d-ada8-4b29-b4a6-a0844738602b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:07.404468 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 68/148] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:07.405145 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578565d-ada8-4b29-b4a6-a0844738602b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:07.416575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:07.513918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578565d-ada8-4b29-b4a6-a0844738602b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:07.548889 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cb916-b43e-42ec-bb57-421ba8836cde None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:07.559702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ebd04-cda9-418e-97d6-a91723c987ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:07.560510 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ebd04-cda9-418e-97d6-a91723c987ce None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:07.560795 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ebd04-cda9-418e-97d6-a91723c987ce None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:07.561069 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ebd04-cda9-418e-97d6-a91723c987ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:07.567377 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ebd04-cda9-418e-97d6-a91723c987ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:07.569092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ebd04-cda9-418e-97d6-a91723c987ce None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:07.570874 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 69/149] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:07.571635 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ebd04-cda9-418e-97d6-a91723c987ce None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:07.598897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ba945-0417-438b-ab06-b6ffcc97b5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:07.599584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ba945-0417-438b-ab06-b6ffcc97b5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:07.599872 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ba945-0417-438b-ab06-b6ffcc97b5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:07.600147 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ba945-0417-438b-ab06-b6ffcc97b5e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:07.600905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389ba945-0417-438b-ab06-b6ffcc97b5e5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:07.601216 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389ba945-0417-438b-ab06-b6ffcc97b5e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:07.602659 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389ba945-0417-438b-ab06-b6ffcc97b5e5 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:07.619860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 81/150] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:07.620692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ba945-0417-438b-ab06-b6ffcc97b5e5 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:07.644959 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:07.655702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:07.680445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ebd04-cda9-418e-97d6-a91723c987ce None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:07.735737 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b378f-87c2-43c3-a281-8e58142ce0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:07.736641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b378f-87c2-43c3-a281-8e58142ce0de None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:07.736924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b378f-87c2-43c3-a281-8e58142ce0de None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:07.737197 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b378f-87c2-43c3-a281-8e58142ce0de None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:07.738455 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b378f-87c2-43c3-a281-8e58142ce0de None admin] RBAC: Authorizing `identity:get_project(project_id=d20d807ab0054a0d994c8b287d262cba)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:07.739931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b378f-87c2-43c3-a281-8e58142ce0de None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:07.741813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 70/151] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:07 2022] GET /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:07.742397 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b378f-87c2-43c3-a281-8e58142ce0de None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:07.796044 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ba945-0417-438b-ab06-b6ffcc97b5e5 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:07.853795 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b378f-87c2-43c3-a281-8e58142ce0de None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:07.871289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5259d-4726-40ef-98be-19f94e79f041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:07.872325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5259d-4726-40ef-98be-19f94e79f041 None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:07.872873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5259d-4726-40ef-98be-19f94e79f041 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:07.873362 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5259d-4726-40ef-98be-19f94e79f041 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:07.874348 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5259d-4726-40ef-98be-19f94e79f041 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:07.876073 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5259d-4726-40ef-98be-19f94e79f041 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:07.894736 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb750383-98b9-4476-ad22-ffa174f71697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:07.895615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb750383-98b9-4476-ad22-ffa174f71697 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:07.895897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb750383-98b9-4476-ad22-ffa174f71697 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:07.896162 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb750383-98b9-4476-ad22-ffa174f71697 None admin] PATH_INFO: `/v3/users/785770e376794f45a62cee26e45bee8f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:07.897593 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb750383-98b9-4476-ad22-ffa174f71697 None admin] RBAC: Authorizing `identity:get_user(user_id=785770e376794f45a62cee26e45bee8f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:07.899060 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb750383-98b9-4476-ad22-ffa174f71697 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:07.900838 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 71/152] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:07 2022] GET /identity/v3/users/785770e376794f45a62cee26e45bee8f => generated 289 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:07.901428 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb750383-98b9-4476-ad22-ffa174f71697 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:07.923266 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:08.001742 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 82/153] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 23 11:25:07 2022] POST /identity/v3/users => generated 283 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:25:08.002596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5259d-4726-40ef-98be-19f94e79f041 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:08.035392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb750383-98b9-4476-ad22-ffa174f71697 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:08.123389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a1bde-4edb-4a9a-8fb8-2997dd415cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:08.124002 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a1bde-4edb-4a9a-8fb8-2997dd415cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:08.124252 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a1bde-4edb-4a9a-8fb8-2997dd415cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:08.124535 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a1bde-4edb-4a9a-8fb8-2997dd415cc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:08.125488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a1bde-4edb-4a9a-8fb8-2997dd415cc6 None admin] RBAC: Adding query filter params (role.id=c9e006de0d6c4812ab29d778e9c8a830, scope.project.id=d20d807ab0054a0d994c8b287d262cba, user.id=d5118162f0624ca794a0f081b0c2cca5) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:08.125803 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a1bde-4edb-4a9a-8fb8-2997dd415cc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:08.127324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a1bde-4edb-4a9a-8fb8-2997dd415cc6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:08.136532 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:08.138185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 72/154] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:25:07 2022] GET /identity/v3/role_assignments?user.id=d5118162f0624ca794a0f081b0c2cca5&scope.project.id=d20d807ab0054a0d994c8b287d262cba&role.id=c9e006de0d6c4812ab29d778e9c8a830 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:08.197198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5259d-4726-40ef-98be-19f94e79f041 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:08.248770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487b6f0-419c-4595-8618-f49f94e50e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:08.249640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b6f0-419c-4595-8618-f49f94e50e68 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:08.250649 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b6f0-419c-4595-8618-f49f94e50e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:08.251012 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b6f0-419c-4595-8618-f49f94e50e68 None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:08.255441 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e487b6f0-419c-4595-8618-f49f94e50e68 None admin] RBAC: Authorizing `identity:get_project(project_id=d20d807ab0054a0d994c8b287d262cba)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:08.256976 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e487b6f0-419c-4595-8618-f49f94e50e68 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:08.259093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 83/155] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:08 2022] GET /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:08.265158 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a1bde-4edb-4a9a-8fb8-2997dd415cc6 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:08.382681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a75f3-5434-4944-8d2c-7b9e5d8a91a4 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:08.382891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a75f3-5434-4944-8d2c-7b9e5d8a91a4 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:08.383293 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:08.384455 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a75f3-5434-4944-8d2c-7b9e5d8a91a4 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:08.386086 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 84/156] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:08.393270 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b0063-02c0-4354-8c77-970b882ba422 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:08.394718 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b0063-02c0-4354-8c77-970b882ba422 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:08.394718 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b0063-02c0-4354-8c77-970b882ba422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:08.415590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a1bde-4edb-4a9a-8fb8-2997dd415cc6 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:08.421567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-7c2b0063-02c0-4354-8c77-970b882ba422 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:08.428807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-7c2b0063-02c0-4354-8c77-970b882ba422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:08.455508 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2eac2-370a-4783-8ff0-31ca9e315ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:08.456443 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2eac2-370a-4783-8ff0-31ca9e315ac9 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:08.456735 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2eac2-370a-4783-8ff0-31ca9e315ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:08.457008 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2eac2-370a-4783-8ff0-31ca9e315ac9 None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba/users/785770e376794f45a62cee26e45bee8f/roles/c8e4f6d521b04ad39661b74575a8b9d3` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:08.459060 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2eac2-370a-4783-8ff0-31ca9e315ac9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d20d807ab0054a0d994c8b287d262cba, user_id=785770e376794f45a62cee26e45bee8f, role_id=c8e4f6d521b04ad39661b74575a8b9d3)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:08.460521 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2eac2-370a-4783-8ff0-31ca9e315ac9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:08.513953 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 85/157] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:08.542293 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b0063-02c0-4354-8c77-970b882ba422 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:08.552772 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 73/158] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:25:08 2022] PUT /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba/users/785770e376794f45a62cee26e45bee8f/roles/c8e4f6d521b04ad39661b74575a8b9d3 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:25:08.617679 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:08.663281 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b0063-02c0-4354-8c77-970b882ba422 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:08.702574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a6d60-a6cc-4950-8ad0-497a1570ce33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:08.704036 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a6d60-a6cc-4950-8ad0-497a1570ce33 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:08.704314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a6d60-a6cc-4950-8ad0-497a1570ce33 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:08.704641 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a6d60-a6cc-4950-8ad0-497a1570ce33 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:08.716854 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a6d60-a6cc-4950-8ad0-497a1570ce33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:08.725735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-1a0a6d60-a6cc-4950-8ad0-497a1570ce33 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:08.725735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:08.725735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:08.725735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:08.725735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:08.725735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:08.725735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:08.725735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:08.725735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:08.725735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:08.725735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:08.725735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:08.725735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:08.725735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:08.725735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:08.725735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:08.726708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:08.726708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:08.726708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:08.726708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:08.726708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:08.726708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:08.726708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:08.726708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:08.726708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:08.726708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:08.726708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:08.726708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:08.726708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:08.726708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:08.726708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:08.726708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:08.727536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:08.727536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:08.727536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:08.727536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:08.727536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:08.727536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:08.727536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:08.727536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:08.727536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:08.727536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:08.727536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:08.727536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:08.727536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 86/159] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:25:08.734307 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2eac2-370a-4783-8ff0-31ca9e315ac9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:08.854078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2eac2-370a-4783-8ff0-31ca9e315ac9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:08.875581 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74567669-6d8e-4069-90f5-ce57dbb75f28 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:08.876188 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74567669-6d8e-4069-90f5-ce57dbb75f28 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:08.877243 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74567669-6d8e-4069-90f5-ce57dbb75f28 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:08.879137 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 87/160] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:08.884888 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355f6e6-7179-40e7-95f2-7f6cc709f1be None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:08.885177 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355f6e6-7179-40e7-95f2-7f6cc709f1be None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:08.885458 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355f6e6-7179-40e7-95f2-7f6cc709f1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:08.912970 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-d355f6e6-7179-40e7-95f2-7f6cc709f1be None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:08.914250 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d355f6e6-7179-40e7-95f2-7f6cc709f1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:08.917695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0052c5b-9a1c-4f63-bf96-f4aadbb67ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:08.918386 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0052c5b-9a1c-4f63-bf96-f4aadbb67ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:08.918701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0052c5b-9a1c-4f63-bf96-f4aadbb67ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:08.918981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0052c5b-9a1c-4f63-bf96-f4aadbb67ef8 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:08.919744 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0052c5b-9a1c-4f63-bf96-f4aadbb67ef8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:08.920057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0052c5b-9a1c-4f63-bf96-f4aadbb67ef8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:08.921461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0052c5b-9a1c-4f63-bf96-f4aadbb67ef8 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:08.925971 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:08.939890 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 74/161] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:08.945081 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0052c5b-9a1c-4f63-bf96-f4aadbb67ef8 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:09.047917 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 88/162] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:09.049884 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65d16d-f8c8-4f44-9b59-daa58ee782c5 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:09.050397 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65d16d-f8c8-4f44-9b59-daa58ee782c5 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:09.050862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65d16d-f8c8-4f44-9b59-daa58ee782c5 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:09.052311 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 89/163] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:09.058974 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd752d-ab46-45d5-9efc-2b30497c9789 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:09.059518 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd752d-ab46-45d5-9efc-2b30497c9789 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:09.059981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd752d-ab46-45d5-9efc-2b30497c9789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:09.084091 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-59bd752d-ab46-45d5-9efc-2b30497c9789 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:09.086390 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-59bd752d-ab46-45d5-9efc-2b30497c9789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:09.125510 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0052c5b-9a1c-4f63-bf96-f4aadbb67ef8 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:09.151116 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:09.174453 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e829cb42-1529-47d8-9578-39f6efb8358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:09.175628 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829cb42-1529-47d8-9578-39f6efb8358b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:09.176050 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829cb42-1529-47d8-9578-39f6efb8358b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:09.176516 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829cb42-1529-47d8-9578-39f6efb8358b None admin] PATH_INFO: `/v3/users/7e575b3126124c04a504aee5827ae3d3` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:09.177648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 90/164] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:09.178177 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e829cb42-1529-47d8-9578-39f6efb8358b None admin] RBAC: Authorizing `identity:get_user(user_id=7e575b3126124c04a504aee5827ae3d3)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:09.178725 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd752d-ab46-45d5-9efc-2b30497c9789 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:09.180036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e829cb42-1529-47d8-9578-39f6efb8358b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:09.181950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 75/165] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:08 2022] GET /identity/v3/users/7e575b3126124c04a504aee5827ae3d3 => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:09.196916 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e829cb42-1529-47d8-9578-39f6efb8358b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:09.294931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:09.326559 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd752d-ab46-45d5-9efc-2b30497c9789 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:09.361545 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e829cb42-1529-47d8-9578-39f6efb8358b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:09.386323 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee7c64-001f-451e-ad65-7c50be3b81c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:09.387169 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee7c64-001f-451e-ad65-7c50be3b81c4 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:09.387448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee7c64-001f-451e-ad65-7c50be3b81c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:09.387720 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee7c64-001f-451e-ad65-7c50be3b81c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:09.412317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee7c64-001f-451e-ad65-7c50be3b81c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:09.420519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-e3ee7c64-001f-451e-ad65-7c50be3b81c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:09.420519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:09.420519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:09.420519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:09.420519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:09.420519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:09.420519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:09.420519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:09.420519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:09.420519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:09.420519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:09.420519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:09.420519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:09.420519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:09.420519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:09.420519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:09.421509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:09.421509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:09.421509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:09.421509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:09.421509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:09.421509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:09.421509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:09.421509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:09.421509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:09.421509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:09.421509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:09.421509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:09.421509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:09.421509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:09.421509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:09.421509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:09.422858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:09.422858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:09.422858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:09.422858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:09.422858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:09.422858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:09.422858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:09.422858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:09.422858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:09.422858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:09.422858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:09.422858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:09.422858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 91/166] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:25:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:09.428801 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee7c64-001f-451e-ad65-7c50be3b81c4 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:09.433317 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cd792-1abc-4674-a153-57bcd4b9e22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:09.433964 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cd792-1abc-4674-a153-57bcd4b9e22c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:09.434245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cd792-1abc-4674-a153-57bcd4b9e22c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:09.434517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cd792-1abc-4674-a153-57bcd4b9e22c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:09.523063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:09.630278 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee7c64-001f-451e-ad65-7c50be3b81c4 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:09.635373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624cd792-1abc-4674-a153-57bcd4b9e22c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:09.636874 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624cd792-1abc-4674-a153-57bcd4b9e22c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:09.684676 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b74df-6810-4943-997c-20e2b74a6c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:09.685503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b74df-6810-4943-997c-20e2b74a6c82 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:09.685787 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b74df-6810-4943-997c-20e2b74a6c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:09.686063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b74df-6810-4943-997c-20e2b74a6c82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:09.702464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b74df-6810-4943-997c-20e2b74a6c82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:09.713583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-225b74df-6810-4943-997c-20e2b74a6c82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:09.713583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:09.713583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:09.713583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:09.713583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:09.713583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:09.713583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:09.713583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:09.713583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:09.713583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:09.713583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:09.713583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:09.713583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:09.713583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:09.713583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:09.713583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:09.714473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:09.714473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:09.714473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:09.714473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:09.714473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:09.714473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:09.714473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:09.714473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:09.714473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:09.714473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:09.714473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:09.714473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:09.714473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:09.714473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:09.714473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:09.714473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:09.720278 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:09.720278 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:09.720278 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:09.720278 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:09.720278 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:09.720278 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:09.720278 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:09.720278 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:09.720278 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:09.720278 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:09.720278 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:09.720278 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:09.722583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 92/167] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:09.723614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b74df-6810-4943-997c-20e2b74a6c82 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:09.831398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:09.915045 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 76/168] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 719 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:09.916006 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cd792-1abc-4674-a153-57bcd4b9e22c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:09.956804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b74df-6810-4943-997c-20e2b74a6c82 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:09.958458 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:10.043915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879ba45-65c9-4f9b-b0d1-36755f57c10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:10.044621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879ba45-65c9-4f9b-b0d1-36755f57c10d None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:10.044906 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879ba45-65c9-4f9b-b0d1-36755f57c10d None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:10.045182 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879ba45-65c9-4f9b-b0d1-36755f57c10d None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:10.045949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879ba45-65c9-4f9b-b0d1-36755f57c10d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:10.046263 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879ba45-65c9-4f9b-b0d1-36755f57c10d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:10.047970 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879ba45-65c9-4f9b-b0d1-36755f57c10d None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:10.056762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cd792-1abc-4674-a153-57bcd4b9e22c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:10.069582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 93/169] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:25:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:10.070460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879ba45-65c9-4f9b-b0d1-36755f57c10d None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:10.106687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bb25a-df1b-41a2-a5e8-a1bcc24790d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:10.107388 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bb25a-df1b-41a2-a5e8-a1bcc24790d9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:10.107666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bb25a-df1b-41a2-a5e8-a1bcc24790d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:10.107939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bb25a-df1b-41a2-a5e8-a1bcc24790d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:10.108860 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2bb25a-df1b-41a2-a5e8-a1bcc24790d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:10.109244 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2bb25a-df1b-41a2-a5e8-a1bcc24790d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:10.110776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2bb25a-df1b-41a2-a5e8-a1bcc24790d9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:10.127544 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 77/170] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:10.128417 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bb25a-df1b-41a2-a5e8-a1bcc24790d9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:10.186929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:10.209465 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879ba45-65c9-4f9b-b0d1-36755f57c10d None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:10.244922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53416fa-5de2-456e-b539-d1f0b8833e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:10.245849 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53416fa-5de2-456e-b539-d1f0b8833e86 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:10.246138 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53416fa-5de2-456e-b539-d1f0b8833e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:10.246473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53416fa-5de2-456e-b539-d1f0b8833e86 None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:10.247784 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53416fa-5de2-456e-b539-d1f0b8833e86 None admin] RBAC: Authorizing `identity:get_project(project_id=d20d807ab0054a0d994c8b287d262cba)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:10.249277 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53416fa-5de2-456e-b539-d1f0b8833e86 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:10.251650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 94/171] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:10 2022] GET /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:10.252447 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53416fa-5de2-456e-b539-d1f0b8833e86 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:10.287933 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:10.306343 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bb25a-df1b-41a2-a5e8-a1bcc24790d9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:10.368586 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53416fa-5de2-456e-b539-d1f0b8833e86 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:10.369427 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d284bc1-df4d-4f8e-87fa-012a21291f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:10.370491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d284bc1-df4d-4f8e-87fa-012a21291f1b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:10.370928 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d284bc1-df4d-4f8e-87fa-012a21291f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:10.371356 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d284bc1-df4d-4f8e-87fa-012a21291f1b None admin] PATH_INFO: `/v3/users/7e575b3126124c04a504aee5827ae3d3` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:10.375337 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d284bc1-df4d-4f8e-87fa-012a21291f1b None admin] RBAC: Authorizing `identity:get_user(user_id=7e575b3126124c04a504aee5827ae3d3)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:10.376950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d284bc1-df4d-4f8e-87fa-012a21291f1b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:10.378774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 78/172] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:10 2022] GET /identity/v3/users/7e575b3126124c04a504aee5827ae3d3 => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:10.379667 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d284bc1-df4d-4f8e-87fa-012a21291f1b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:10.403932 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b3b01-be8c-4db1-a315-87c8838b8e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:10.405087 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b3b01-be8c-4db1-a315-87c8838b8e1b None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:10.405203 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b3b01-be8c-4db1-a315-87c8838b8e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:10.410512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b3b01-be8c-4db1-a315-87c8838b8e1b None admin] PATH_INFO: `/v3/users/7e575b3126124c04a504aee5827ae3d3` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:10.410512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b3b01-be8c-4db1-a315-87c8838b8e1b None admin] RBAC: Authorizing `identity:get_user(user_id=7e575b3126124c04a504aee5827ae3d3)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:10.410512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b3b01-be8c-4db1-a315-87c8838b8e1b None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:10.410512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 95/173] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:10 2022] GET /identity/v3/users/7e575b3126124c04a504aee5827ae3d3 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:10.411320 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b3b01-be8c-4db1-a315-87c8838b8e1b None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:10.528164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:10.532303 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b3b01-be8c-4db1-a315-87c8838b8e1b None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:10.547561 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:10.559115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d284bc1-df4d-4f8e-87fa-012a21291f1b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:10.600750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559def5f-3801-4c9b-8228-ff238e3ff9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:10.601406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559def5f-3801-4c9b-8228-ff238e3ff9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:10.601693 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559def5f-3801-4c9b-8228-ff238e3ff9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:10.601969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559def5f-3801-4c9b-8228-ff238e3ff9b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:10.624032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ab089-658b-4c8d-a704-e1562ef7bd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:10.624743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ab089-658b-4c8d-a704-e1562ef7bd5b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:10.625022 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ab089-658b-4c8d-a704-e1562ef7bd5b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:10.625302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ab089-658b-4c8d-a704-e1562ef7bd5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:10.626208 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ab089-658b-4c8d-a704-e1562ef7bd5b None admin] RBAC: Adding query filter params (role.id=75f08cd0671f436c9f968976aa8baa7f, scope.project.id=d20d807ab0054a0d994c8b287d262cba, user.id=7e575b3126124c04a504aee5827ae3d3) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:10.626541 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ab089-658b-4c8d-a704-e1562ef7bd5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:10.630426 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ab089-658b-4c8d-a704-e1562ef7bd5b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:10.641301 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 79/174] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:25:10 2022] GET /identity/v3/role_assignments?user.id=7e575b3126124c04a504aee5827ae3d3&scope.project.id=d20d807ab0054a0d994c8b287d262cba&role.id=75f08cd0671f436c9f968976aa8baa7f => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:10.642206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ab089-658b-4c8d-a704-e1562ef7bd5b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:10.695346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559def5f-3801-4c9b-8228-ff238e3ff9b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:10.696846 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559def5f-3801-4c9b-8228-ff238e3ff9b6 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:10.804862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:10.808572 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 96/175] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:10.814295 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559def5f-3801-4c9b-8228-ff238e3ff9b6 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:10.868955 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ab089-658b-4c8d-a704-e1562ef7bd5b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:10.924926 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559def5f-3801-4c9b-8228-ff238e3ff9b6 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:10.972720 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ef613-e618-4aeb-8856-eb2c22316841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:10.973612 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ef613-e618-4aeb-8856-eb2c22316841 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:10.973899 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ef613-e618-4aeb-8856-eb2c22316841 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:10.974178 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ef613-e618-4aeb-8856-eb2c22316841 None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:10.976235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e9bb5-178e-492f-8035-21ab4bbb365f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:10.976903 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e9bb5-178e-492f-8035-21ab4bbb365f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:10.978065 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e9bb5-178e-492f-8035-21ab4bbb365f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:10.978354 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e9bb5-178e-492f-8035-21ab4bbb365f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:10.979264 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ef613-e618-4aeb-8856-eb2c22316841 None admin] RBAC: Authorizing `identity:get_project(project_id=d20d807ab0054a0d994c8b287d262cba)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:10.980757 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ef613-e618-4aeb-8856-eb2c22316841 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:10.982904 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 97/176] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:10 2022] GET /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:10.989213 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ef613-e618-4aeb-8856-eb2c22316841 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:11.062436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:11.122938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:11.163439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ef613-e618-4aeb-8856-eb2c22316841 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:11.183392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35e9bb5-178e-492f-8035-21ab4bbb365f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:11.184885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35e9bb5-178e-492f-8035-21ab4bbb365f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:11.199412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66908d12-c368-4276-8e73-112e5d0b6898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:11.200080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66908d12-c368-4276-8e73-112e5d0b6898 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:11.200369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66908d12-c368-4276-8e73-112e5d0b6898 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:11.200656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66908d12-c368-4276-8e73-112e5d0b6898 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:11.201502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66908d12-c368-4276-8e73-112e5d0b6898 None admin] RBAC: Adding query filter params (role.id=25e99b2a08914aa0ab18cc4955f25364, scope.project.id=d20d807ab0054a0d994c8b287d262cba, user.id=7e575b3126124c04a504aee5827ae3d3) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:11.201822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66908d12-c368-4276-8e73-112e5d0b6898 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:11.203606 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66908d12-c368-4276-8e73-112e5d0b6898 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:11.209606 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 98/177] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:25:10 2022] GET /identity/v3/role_assignments?user.id=7e575b3126124c04a504aee5827ae3d3&scope.project.id=d20d807ab0054a0d994c8b287d262cba&role.id=25e99b2a08914aa0ab18cc4955f25364 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:11.380693 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 80/178] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 739 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:11.386563 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66908d12-c368-4276-8e73-112e5d0b6898 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:11.518513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66908d12-c368-4276-8e73-112e5d0b6898 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:11.550920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:11.555502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476c6af-0756-43c8-b223-51bcb1cf92db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:11.556375 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476c6af-0756-43c8-b223-51bcb1cf92db None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:11.556659 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476c6af-0756-43c8-b223-51bcb1cf92db None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:11.556934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476c6af-0756-43c8-b223-51bcb1cf92db None admin] PATH_INFO: `/v3/projects/740b5771a1034d5f986d75ce345e365b` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:11.563596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476c6af-0756-43c8-b223-51bcb1cf92db None admin] RBAC: Authorizing `identity:get_project(project_id=740b5771a1034d5f986d75ce345e365b)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:11.563596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476c6af-0756-43c8-b223-51bcb1cf92db None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:11.563596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 99/179] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:11 2022] GET /identity/v3/projects/740b5771a1034d5f986d75ce345e365b => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:11.565589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c531c9-4197-4002-afa7-93295e2ae79e None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:11.565879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c531c9-4197-4002-afa7-93295e2ae79e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:11.566167 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c531c9-4197-4002-afa7-93295e2ae79e None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:11.567674 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 81/180] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:11.569050 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476c6af-0756-43c8-b223-51bcb1cf92db None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:11.573129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca976d7-e94b-40ea-93df-d0952bbc0581 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:11.573416 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca976d7-e94b-40ea-93df-d0952bbc0581 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:11.573689 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca976d7-e94b-40ea-93df-d0952bbc0581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:11.594558 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-6ca976d7-e94b-40ea-93df-d0952bbc0581 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:11.594771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6ca976d7-e94b-40ea-93df-d0952bbc0581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:11.685910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476c6af-0756-43c8-b223-51bcb1cf92db None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:11.698931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:11.715657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 82/181] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:25:11.723444 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab173f9b-d4ac-4302-adb5-8a36a2ddcb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:11.724842 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab173f9b-d4ac-4302-adb5-8a36a2ddcb01 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:11.725340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab173f9b-d4ac-4302-adb5-8a36a2ddcb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:11.725836 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab173f9b-d4ac-4302-adb5-8a36a2ddcb01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:11.726945 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab173f9b-d4ac-4302-adb5-8a36a2ddcb01 None admin] RBAC: Adding query filter params (role.id=75f08cd0671f436c9f968976aa8baa7f, scope.project.id=740b5771a1034d5f986d75ce345e365b, user.id=7e575b3126124c04a504aee5827ae3d3) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:11.727464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab173f9b-d4ac-4302-adb5-8a36a2ddcb01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:11.729165 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab173f9b-d4ac-4302-adb5-8a36a2ddcb01 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:11.736375 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 100/182] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:25:11 2022] GET /identity/v3/role_assignments?user.id=7e575b3126124c04a504aee5827ae3d3&scope.project.id=740b5771a1034d5f986d75ce345e365b&role.id=75f08cd0671f436c9f968976aa8baa7f => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:11.759550 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca976d7-e94b-40ea-93df-d0952bbc0581 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:11.919187 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca976d7-e94b-40ea-93df-d0952bbc0581 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:11.961266 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58150c3b-1d61-429c-b942-9b001003ce93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:11.962001 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58150c3b-1d61-429c-b942-9b001003ce93 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:11.962288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58150c3b-1d61-429c-b942-9b001003ce93 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:11.962587 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58150c3b-1d61-429c-b942-9b001003ce93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:11.975056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58150c3b-1d61-429c-b942-9b001003ce93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:11.980748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823fd6c-cbff-493a-adb2-75c7fc7a088b None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:11.981069 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823fd6c-cbff-493a-adb2-75c7fc7a088b None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:11.981350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823fd6c-cbff-493a-adb2-75c7fc7a088b None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:11.982751 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 101/183] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:25:11.986074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-58150c3b-1d61-429c-b942-9b001003ce93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:25:11.986074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:11.986074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:11.986074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:11.986074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:11.986074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:11.986074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:11.986074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:11.986074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:11.986074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:11.986074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:11.986074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:11.986074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:11.986074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:11.986074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:11.986074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:11.986940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:11.986940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:11.986940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:11.986940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:11.986940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:11.986940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:11.986940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:11.986940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:11.986940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:11.986940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:11.986940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:11.986940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:11.986940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:11.986940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:11.986940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:11.986940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:11.987722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:11.987722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:11.987722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:11.987722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:11.987722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:11.987722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:11.987722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:11.987722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:11.987722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:11.987722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:11.987722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:25:11.987722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:11.987722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 83/184] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:25:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:11.988845 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34a1ed-d112-4da0-a0de-2848eb657c83 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:11.994562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34a1ed-d112-4da0-a0de-2848eb657c83 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:11.994562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34a1ed-d112-4da0-a0de-2848eb657c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:11.994917 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58150c3b-1d61-429c-b942-9b001003ce93 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:12.011894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-5b34a1ed-d112-4da0-a0de-2848eb657c83 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:12.013261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-5b34a1ed-d112-4da0-a0de-2848eb657c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:12.053455 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:12.090171 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:12.102195 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 102/185] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:12.103559 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdcc1b-3d3a-4b1f-b2e5-7935a092bfbd None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:12.103849 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdcc1b-3d3a-4b1f-b2e5-7935a092bfbd None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:12.104507 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdcc1b-3d3a-4b1f-b2e5-7935a092bfbd None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:12.105827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 103/186] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:12.111785 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2f48e-a0d6-48f9-b0fd-fff1ec411432 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:12.111978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2f48e-a0d6-48f9-b0fd-fff1ec411432 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:12.112302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2f48e-a0d6-48f9-b0fd-fff1ec411432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:12.117476 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58150c3b-1d61-429c-b942-9b001003ce93 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:12.133939 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-35c2f48e-a0d6-48f9-b0fd-fff1ec411432 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:12.134148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-35c2f48e-a0d6-48f9-b0fd-fff1ec411432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:12.158199 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e9137-8012-4f44-8cd7-c5dd213f5c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:12.159217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e9137-8012-4f44-8cd7-c5dd213f5c1f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:12.159674 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e9137-8012-4f44-8cd7-c5dd213f5c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:12.160191 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e9137-8012-4f44-8cd7-c5dd213f5c1f None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:12.161311 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e9137-8012-4f44-8cd7-c5dd213f5c1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:12.161786 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e9137-8012-4f44-8cd7-c5dd213f5c1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:12.163647 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e9137-8012-4f44-8cd7-c5dd213f5c1f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:12.173073 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 84/187] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:12.177767 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e9137-8012-4f44-8cd7-c5dd213f5c1f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:12.207802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 104/188] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:25:12.219204 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2f48e-a0d6-48f9-b0fd-fff1ec411432 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:12.291945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e9137-8012-4f44-8cd7-c5dd213f5c1f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:12.305832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:12.314385 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2f48e-a0d6-48f9-b0fd-fff1ec411432 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:12.332709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f13fd6-ed70-48d8-9645-8fb9c957d6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:12.333897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f13fd6-ed70-48d8-9645-8fb9c957d6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:12.334181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f13fd6-ed70-48d8-9645-8fb9c957d6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:12.334455 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f13fd6-ed70-48d8-9645-8fb9c957d6b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:12.340012 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:12.351206 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d2a38-07e0-4e4b-9a28-84065b14d714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:12.352081 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d2a38-07e0-4e4b-9a28-84065b14d714 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:12.352363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d2a38-07e0-4e4b-9a28-84065b14d714 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:12.352634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d2a38-07e0-4e4b-9a28-84065b14d714 None admin] PATH_INFO: `/v3/users/159c476f43a84914883a4ee264db9ff9` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:12.352814 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f13fd6-ed70-48d8-9645-8fb9c957d6b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:12.362910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-85f13fd6-ed70-48d8-9645-8fb9c957d6b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:25:12.362910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:12.362910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:12.362910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:12.362910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:12.362910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:12.362910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:12.362910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:12.362910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:12.362910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:12.362910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:12.362910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:12.362910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:12.362910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:12.362910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:12.362910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:12.364210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:12.364210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:12.364210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:12.364210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:12.364210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:12.364210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:12.364210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:12.364210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:12.364210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:12.364210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:12.364210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:12.364210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:12.364210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:12.364210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:12.364210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:12.364210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:12.364989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:12.364989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:12.364989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:12.364989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:12.364989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:12.364989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:12.364989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:12.364989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:12.364989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:12.364989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:12.364989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:25:12.364989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:12.365571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 85/189] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:12.371046 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f13fd6-ed70-48d8-9645-8fb9c957d6b4 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:12.373931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d2a38-07e0-4e4b-9a28-84065b14d714 None admin] RBAC: Authorizing `identity:get_user(user_id=159c476f43a84914883a4ee264db9ff9)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:12.376140 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d2a38-07e0-4e4b-9a28-84065b14d714 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:12.378864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 105/190] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:12 2022] GET /identity/v3/users/159c476f43a84914883a4ee264db9ff9 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:12.379557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d2a38-07e0-4e4b-9a28-84065b14d714 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:12.486771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f13fd6-ed70-48d8-9645-8fb9c957d6b4 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:12.491523 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d2a38-07e0-4e4b-9a28-84065b14d714 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:12.520516 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb58ba2-e7d6-4214-a43c-c5fff2ae7bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:12.521325 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb58ba2-e7d6-4214-a43c-c5fff2ae7bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:12.521610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb58ba2-e7d6-4214-a43c-c5fff2ae7bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:12.521893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb58ba2-e7d6-4214-a43c-c5fff2ae7bf9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:12.526223 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9117f5-fe1e-4e97-9015-e9cd5e8f8050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:12.526915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9117f5-fe1e-4e97-9015-e9cd5e8f8050 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:12.527190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9117f5-fe1e-4e97-9015-e9cd5e8f8050 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:12.527461 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9117f5-fe1e-4e97-9015-e9cd5e8f8050 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:12.528196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9117f5-fe1e-4e97-9015-e9cd5e8f8050 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:12.528502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9117f5-fe1e-4e97-9015-e9cd5e8f8050 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:12.529878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9117f5-fe1e-4e97-9015-e9cd5e8f8050 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:12.531878 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb58ba2-e7d6-4214-a43c-c5fff2ae7bf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:12.538129 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 106/191] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:12.538952 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9117f5-fe1e-4e97-9015-e9cd5e8f8050 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:12.540079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-aeb58ba2-e7d6-4214-a43c-c5fff2ae7bf9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:12.540079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:12.540079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:12.540079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:12.540079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:12.540079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:12.540079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:12.540079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:12.540079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:12.540079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:12.540079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:12.540079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:12.540079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:12.540079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:12.540079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:12.540079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:12.540952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:12.540952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:12.540952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:12.540952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:12.540952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:12.540952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:12.540952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:12.540952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:12.540952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:12.540952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:12.540952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:12.540952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:12.540952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:12.540952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:12.540952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:12.540952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:12.541746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:12.541746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:12.541746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:12.541746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:12.541746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:12.541746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:12.541746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:12.541746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:12.541746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:12.541746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:12.541746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:12.541746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:12.541746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 86/192] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:25:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:12.542887 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b45d99-1e90-4f69-a8aa-9569040370aa None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:12.542995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b45d99-1e90-4f69-a8aa-9569040370aa None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:12.543241 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b45d99-1e90-4f69-a8aa-9569040370aa None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:12.544572 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 87/193] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:12.547144 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b45d99-1e90-4f69-a8aa-9569040370aa None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:12.577007 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:12.590138 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:12.645908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9117f5-fe1e-4e97-9015-e9cd5e8f8050 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:12.648414 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b45d99-1e90-4f69-a8aa-9569040370aa None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:12.680109 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba601ac-5fe9-48ab-90a5-b7100ef88e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:12.680759 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba601ac-5fe9-48ab-90a5-b7100ef88e82 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:12.681051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba601ac-5fe9-48ab-90a5-b7100ef88e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:12.681327 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba601ac-5fe9-48ab-90a5-b7100ef88e82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:12.681946 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce3397-5cff-42f1-8aa3-780c225f5a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:12.682855 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce3397-5cff-42f1-8aa3-780c225f5a1d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:12.683136 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce3397-5cff-42f1-8aa3-780c225f5a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:12.683409 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce3397-5cff-42f1-8aa3-780c225f5a1d None admin] PATH_INFO: `/v3/users/785770e376794f45a62cee26e45bee8f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:12.684599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce3397-5cff-42f1-8aa3-780c225f5a1d None admin] RBAC: Authorizing `identity:get_user(user_id=785770e376794f45a62cee26e45bee8f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:12.686049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce3397-5cff-42f1-8aa3-780c225f5a1d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:12.688293 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 88/194] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:12 2022] GET /identity/v3/users/785770e376794f45a62cee26e45bee8f => generated 289 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:12.689062 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce3397-5cff-42f1-8aa3-780c225f5a1d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:12.751998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba601ac-5fe9-48ab-90a5-b7100ef88e82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:12.753703 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba601ac-5fe9-48ab-90a5-b7100ef88e82 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:12.786710 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:12.789590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce3397-5cff-42f1-8aa3-780c225f5a1d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:12.797847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:12.825189 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11967df4-60bf-4d69-aaeb-74459b512a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:12.825858 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11967df4-60bf-4d69-aaeb-74459b512a4f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:12.826132 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11967df4-60bf-4d69-aaeb-74459b512a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:12.826402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11967df4-60bf-4d69-aaeb-74459b512a4f None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:12.827175 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11967df4-60bf-4d69-aaeb-74459b512a4f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:12.827482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11967df4-60bf-4d69-aaeb-74459b512a4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:12.828883 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11967df4-60bf-4d69-aaeb-74459b512a4f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:12.836796 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 89/195] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:25:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:12.838069 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4f7a7-d4bf-420f-9562-f7c4d7c3b1f6 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:12.838368 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4f7a7-d4bf-420f-9562-f7c4d7c3b1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:12.838676 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4f7a7-d4bf-420f-9562-f7c4d7c3b1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:12.852814 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 107/196] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:12.853649 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba601ac-5fe9-48ab-90a5-b7100ef88e82 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:12.855321 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-41e4f7a7-d4bf-420f-9562-f7c4d7c3b1f6 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:12.856512 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-41e4f7a7-d4bf-420f-9562-f7c4d7c3b1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:12.922393 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 90/197] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:12.923435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4f7a7-d4bf-420f-9562-f7c4d7c3b1f6 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:12.954351 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba601ac-5fe9-48ab-90a5-b7100ef88e82 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:12.996425 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae732a-39ac-429c-80e8-9b8bc48ef4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:12.997076 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae732a-39ac-429c-80e8-9b8bc48ef4ef None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:12.997373 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae732a-39ac-429c-80e8-9b8bc48ef4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:12.997642 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae732a-39ac-429c-80e8-9b8bc48ef4ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:13.011728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:13.014261 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:13.023040 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4f7a7-d4bf-420f-9562-f7c4d7c3b1f6 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:13.057537 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07dd6e-60df-4fce-bade-e84393d5dbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:13.058409 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07dd6e-60df-4fce-bade-e84393d5dbb2 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:13.058719 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07dd6e-60df-4fce-bade-e84393d5dbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:13.058989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07dd6e-60df-4fce-bade-e84393d5dbb2 None admin] PATH_INFO: `/v3/users/159c476f43a84914883a4ee264db9ff9` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:13.060206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07dd6e-60df-4fce-bade-e84393d5dbb2 None admin] RBAC: Authorizing `identity:get_user(user_id=159c476f43a84914883a4ee264db9ff9)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:13.061652 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07dd6e-60df-4fce-bade-e84393d5dbb2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:13.063967 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 91/198] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:12 2022] GET /identity/v3/users/159c476f43a84914883a4ee264db9ff9 => generated 283 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:13.068837 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07dd6e-60df-4fce-bade-e84393d5dbb2 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:13.071607 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae732a-39ac-429c-80e8-9b8bc48ef4ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:13.073105 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae732a-39ac-429c-80e8-9b8bc48ef4ef None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:13.166471 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07dd6e-60df-4fce-bade-e84393d5dbb2 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:13.172827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 108/199] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:13.173652 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae732a-39ac-429c-80e8-9b8bc48ef4ef None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:13.205388 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb999e0-cb2c-420d-8aef-36d7fca14f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:13.206245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb999e0-cb2c-420d-8aef-36d7fca14f18 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:13.206554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb999e0-cb2c-420d-8aef-36d7fca14f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:13.206836 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb999e0-cb2c-420d-8aef-36d7fca14f18 None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:13.208021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb999e0-cb2c-420d-8aef-36d7fca14f18 None admin] RBAC: Authorizing `identity:get_project(project_id=b05339ac8c27421a9a1921726dcbdc7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:13.209485 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb999e0-cb2c-420d-8aef-36d7fca14f18 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:13.211236 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 92/200] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:13 2022] GET /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:13.211994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb999e0-cb2c-420d-8aef-36d7fca14f18 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:13.219069 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:13.243647 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:13.273358 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae732a-39ac-429c-80e8-9b8bc48ef4ef None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:13.306292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b570a86-2cd1-4aba-ab02-539c096623b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:13.307096 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b570a86-2cd1-4aba-ab02-539c096623b7 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:13.307384 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b570a86-2cd1-4aba-ab02-539c096623b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:13.307664 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b570a86-2cd1-4aba-ab02-539c096623b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:13.308839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b570a86-2cd1-4aba-ab02-539c096623b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:13.310309 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b570a86-2cd1-4aba-ab02-539c096623b7 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:13.311956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 109/201] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:13.315034 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb999e0-cb2c-420d-8aef-36d7fca14f18 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:13.317124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b570a86-2cd1-4aba-ab02-539c096623b7 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:13.350630 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f552b50-4c38-44e7-9754-0acdd31faaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:13.351286 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f552b50-4c38-44e7-9754-0acdd31faaa4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:13.351570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f552b50-4c38-44e7-9754-0acdd31faaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:13.351841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f552b50-4c38-44e7-9754-0acdd31faaa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:13.415153 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b570a86-2cd1-4aba-ab02-539c096623b7 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:13.417772 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f552b50-4c38-44e7-9754-0acdd31faaa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:13.419284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f552b50-4c38-44e7-9754-0acdd31faaa4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:13.427650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:13.445788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:13.448839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1da82-382e-4ef7-b7c4-f5ba94ae8606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:13.449756 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1da82-382e-4ef7-b7c4-f5ba94ae8606 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:13.450039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1da82-382e-4ef7-b7c4-f5ba94ae8606 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:13.450314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1da82-382e-4ef7-b7c4-f5ba94ae8606 None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:13.451519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e1da82-382e-4ef7-b7c4-f5ba94ae8606 None admin] RBAC: Authorizing `identity:get_project(project_id=d20d807ab0054a0d994c8b287d262cba)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:13.453453 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e1da82-382e-4ef7-b7c4-f5ba94ae8606 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:13.455518 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 110/202] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:13 2022] GET /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:13.456581 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1da82-382e-4ef7-b7c4-f5ba94ae8606 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:13.520658 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 93/203] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:13.520658 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f552b50-4c38-44e7-9754-0acdd31faaa4 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:13.563988 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1da82-382e-4ef7-b7c4-f5ba94ae8606 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:13.596839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b08721-9899-4c45-b083-36f6f528c1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:13.597495 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b08721-9899-4c45-b083-36f6f528c1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:13.597780 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b08721-9899-4c45-b083-36f6f528c1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:13.598055 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b08721-9899-4c45-b083-36f6f528c1a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:13.598942 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b08721-9899-4c45-b083-36f6f528c1a2 None admin] RBAC: Adding query filter params (role.id=c9e006de0d6c4812ab29d778e9c8a830, scope.project.id=b05339ac8c27421a9a1921726dcbdc7f, user.id=159c476f43a84914883a4ee264db9ff9) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:13.599256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b08721-9899-4c45-b083-36f6f528c1a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:13.600694 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b08721-9899-4c45-b083-36f6f528c1a2 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:13.606009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 111/204] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:25:13 2022] GET /identity/v3/role_assignments?user.id=159c476f43a84914883a4ee264db9ff9&scope.project.id=b05339ac8c27421a9a1921726dcbdc7f&role.id=c9e006de0d6c4812ab29d778e9c8a830 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:13.606810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b08721-9899-4c45-b083-36f6f528c1a2 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:13.632132 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f552b50-4c38-44e7-9754-0acdd31faaa4 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:13.663709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:13.670133 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:13.671724 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c57f22-ebe9-43e4-b17f-327185970126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:13.672556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c57f22-ebe9-43e4-b17f-327185970126 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:13.672840 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c57f22-ebe9-43e4-b17f-327185970126 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:13.673117 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c57f22-ebe9-43e4-b17f-327185970126 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:13.673855 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c57f22-ebe9-43e4-b17f-327185970126 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:13.675387 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c57f22-ebe9-43e4-b17f-327185970126 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:13.737810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b08721-9899-4c45-b083-36f6f528c1a2 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:13.771256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3990a-989d-4f28-abb9-8dfb0a208031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:13.771911 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3990a-989d-4f28-abb9-8dfb0a208031 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:13.772190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3990a-989d-4f28-abb9-8dfb0a208031 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:13.772461 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3990a-989d-4f28-abb9-8dfb0a208031 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:13.773317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3990a-989d-4f28-abb9-8dfb0a208031 None admin] RBAC: Adding query filter params (role.id=c8e4f6d521b04ad39661b74575a8b9d3, scope.project.id=d20d807ab0054a0d994c8b287d262cba, user.id=785770e376794f45a62cee26e45bee8f) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:13.773627 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3990a-989d-4f28-abb9-8dfb0a208031 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:13.775111 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3990a-989d-4f28-abb9-8dfb0a208031 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:13.790761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 112/205] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:25:13 2022] GET /identity/v3/role_assignments?user.id=785770e376794f45a62cee26e45bee8f&scope.project.id=d20d807ab0054a0d994c8b287d262cba&role.id=c8e4f6d521b04ad39661b74575a8b9d3 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:13.790761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3990a-989d-4f28-abb9-8dfb0a208031 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:13.791287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 94/206] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 23 11:25:13 2022] POST /identity/v3/users => generated 297 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:25:13.927604 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3990a-989d-4f28-abb9-8dfb0a208031 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:13.944439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:13.971157 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e32b59-6f51-4bc7-89d8-a82229b40a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:13.972388 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e32b59-6f51-4bc7-89d8-a82229b40a75 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:13.972900 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e32b59-6f51-4bc7-89d8-a82229b40a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:13.973189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e32b59-6f51-4bc7-89d8-a82229b40a75 None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:13.974453 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e32b59-6f51-4bc7-89d8-a82229b40a75 None admin] RBAC: Authorizing `identity:get_project(project_id=b05339ac8c27421a9a1921726dcbdc7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:13.975962 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e32b59-6f51-4bc7-89d8-a82229b40a75 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:13.977885 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 113/207] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:13 2022] GET /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:13.983835 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c57f22-ebe9-43e4-b17f-327185970126 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:14.102867 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c57f22-ebe9-43e4-b17f-327185970126 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:14.142111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d989a5-981e-46ca-b7a1-7b29ecbab406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:14.142848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d989a5-981e-46ca-b7a1-7b29ecbab406 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:14.143130 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d989a5-981e-46ca-b7a1-7b29ecbab406 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:14.143398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d989a5-981e-46ca-b7a1-7b29ecbab406 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:14.144243 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d989a5-981e-46ca-b7a1-7b29ecbab406 None admin] RBAC: Adding query filter params (role.id=25e99b2a08914aa0ab18cc4955f25364, scope.project.id=b05339ac8c27421a9a1921726dcbdc7f, user.id=159c476f43a84914883a4ee264db9ff9) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:14.144549 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d989a5-981e-46ca-b7a1-7b29ecbab406 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:14.145988 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d989a5-981e-46ca-b7a1-7b29ecbab406 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:14.153060 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 95/208] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:25:13 2022] GET /identity/v3/role_assignments?user.id=159c476f43a84914883a4ee264db9ff9&scope.project.id=b05339ac8c27421a9a1921726dcbdc7f&role.id=25e99b2a08914aa0ab18cc4955f25364 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:15.669464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd607306-3751-4116-84cc-94d400bb644e None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:15.669977 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd607306-3751-4116-84cc-94d400bb644e None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:15.670050 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd607306-3751-4116-84cc-94d400bb644e None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:15.672240 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 114/209] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:15.681118 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a5e34-9fce-4ddc-a2de-a7b60e5fd390 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:15.682402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a5e34-9fce-4ddc-a2de-a7b60e5fd390 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:15.683211 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a5e34-9fce-4ddc-a2de-a7b60e5fd390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:15.711797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-644a5e34-9fce-4ddc-a2de-a7b60e5fd390 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:15.714204 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-644a5e34-9fce-4ddc-a2de-a7b60e5fd390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:15.788131 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:15.817379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 96/210] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:15.841458 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd607306-3751-4116-84cc-94d400bb644e None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:15.958312 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd607306-3751-4116-84cc-94d400bb644e None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:15.997294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d7e3b-2957-47f5-bbe4-4a5c53efc236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:15.998555 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d7e3b-2957-47f5-bbe4-4a5c53efc236 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:15.999061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d7e3b-2957-47f5-bbe4-4a5c53efc236 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:15.999554 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d7e3b-2957-47f5-bbe4-4a5c53efc236 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:16.011927 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d7e3b-2957-47f5-bbe4-4a5c53efc236 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:16.020721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-fa8d7e3b-2957-47f5-bbe4-4a5c53efc236 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:25:16.020721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:16.020721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:16.020721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:16.020721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:16.020721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:16.020721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:16.020721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:16.020721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:16.020721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:16.020721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:16.020721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:16.020721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:16.020721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:16.020721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:16.020721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:16.021644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:16.021644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:16.021644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:16.021644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:16.021644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:16.021644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:16.021644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:16.021644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:16.021644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:16.021644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:16.021644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:16.021644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:16.021644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:16.021644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:16.021644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:16.021644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:16.022446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:16.022446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:16.022446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:16.022446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:16.022446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:16.022446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:16.022446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:16.022446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:16.022446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:16.022446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:16.022446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:25:16.022446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:16.024915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 115/211] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:25:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:16.033654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a5e34-9fce-4ddc-a2de-a7b60e5fd390 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:16.131028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bec711-e7c9-4183-827c-303f4e8a2e84 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:16.131674 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bec711-e7c9-4183-827c-303f4e8a2e84 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:16.132251 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bec711-e7c9-4183-827c-303f4e8a2e84 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:16.133891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 116/212] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:25:16.140849 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708aedbb-c70f-4f68-9211-0749785d775f None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:16.141137 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708aedbb-c70f-4f68-9211-0749785d775f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:16.141416 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708aedbb-c70f-4f68-9211-0749785d775f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:16.159510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-708aedbb-c70f-4f68-9211-0749785d775f None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:16.160732 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-708aedbb-c70f-4f68-9211-0749785d775f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:16.165374 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a5e34-9fce-4ddc-a2de-a7b60e5fd390 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:16.201389 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:16.211632 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40694a45-770d-4939-a12a-97bd3a0169e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:16.219033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40694a45-770d-4939-a12a-97bd3a0169e6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:16.219033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40694a45-770d-4939-a12a-97bd3a0169e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:16.219033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40694a45-770d-4939-a12a-97bd3a0169e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:16.219033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40694a45-770d-4939-a12a-97bd3a0169e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:16.219033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40694a45-770d-4939-a12a-97bd3a0169e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:16.219033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40694a45-770d-4939-a12a-97bd3a0169e6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:16.226601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 97/213] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:16.228804 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d970ff0-f649-477f-8b33-45415add2d41 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:16.229305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d970ff0-f649-477f-8b33-45415add2d41 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:16.229779 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d970ff0-f649-477f-8b33-45415add2d41 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:16.231838 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 98/214] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:16.236460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d970ff0-f649-477f-8b33-45415add2d41 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:16.247710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 117/215] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:25:16.248804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae987c2c-6a58-4e26-b398-6758482e3aeb None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:16.249097 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae987c2c-6a58-4e26-b398-6758482e3aeb None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:16.249376 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae987c2c-6a58-4e26-b398-6758482e3aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:16.273504 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-ae987c2c-6a58-4e26-b398-6758482e3aeb None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:16.274623 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-ae987c2c-6a58-4e26-b398-6758482e3aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:16.286787 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:16.347291 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 118/216] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:16.348434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae987c2c-6a58-4e26-b398-6758482e3aeb None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:16.356911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d970ff0-f649-477f-8b33-45415add2d41 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:16.392807 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa26101-73f8-44ed-80ee-b71a2d56bb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:16.393606 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa26101-73f8-44ed-80ee-b71a2d56bb3a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:16.393887 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa26101-73f8-44ed-80ee-b71a2d56bb3a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:16.394166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa26101-73f8-44ed-80ee-b71a2d56bb3a None admin] PATH_INFO: `/v3/users/159c476f43a84914883a4ee264db9ff9` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:16.395391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa26101-73f8-44ed-80ee-b71a2d56bb3a None admin] RBAC: Authorizing `identity:get_user(user_id=159c476f43a84914883a4ee264db9ff9)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:16.396845 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa26101-73f8-44ed-80ee-b71a2d56bb3a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:16.398522 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 99/217] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:16 2022] GET /identity/v3/users/159c476f43a84914883a4ee264db9ff9 => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:16.399741 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e357b8a-ec84-4f1e-a3de-0572af42a51f None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:16.400026 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e357b8a-ec84-4f1e-a3de-0572af42a51f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:16.400303 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e357b8a-ec84-4f1e-a3de-0572af42a51f None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:16.401532 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 100/218] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:16.402248 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e357b8a-ec84-4f1e-a3de-0572af42a51f None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:16.466455 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:16.473289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae987c2c-6a58-4e26-b398-6758482e3aeb None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:16.510381 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2cfb0-4c42-4a66-9415-cab7b95808ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:16.511228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2cfb0-4c42-4a66-9415-cab7b95808ca None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:16.511515 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2cfb0-4c42-4a66-9415-cab7b95808ca None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:16.511790 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2cfb0-4c42-4a66-9415-cab7b95808ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:16.513214 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e357b8a-ec84-4f1e-a3de-0572af42a51f None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:16.522345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2cfb0-4c42-4a66-9415-cab7b95808ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:16.522887 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:16.530035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-33b2cfb0-4c42-4a66-9415-cab7b95808ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:16.530035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:16.530035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:16.530035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:16.530035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:16.530035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:16.530035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:16.530035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:16.530035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:16.530035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:16.530035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:16.530035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:16.530035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:16.530035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:16.530035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:16.530035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:16.539152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:16.539152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:16.539152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:16.539152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:16.539152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:16.539152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:16.539152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:16.539152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:16.539152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:16.539152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:16.539152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:16.539152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:16.539152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:16.539152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:16.539152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:16.539152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:16.540084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:16.540084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:16.540084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:16.540084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:16.540084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:16.540084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:16.540084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:16.540084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:16.540084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:16.540084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:16.540084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:16.540084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:16.540084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 119/219] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:16.540084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2cfb0-4c42-4a66-9415-cab7b95808ca None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:16.550761 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f41890-f03a-4730-9030-f37a9897be06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:16.551557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f41890-f03a-4730-9030-f37a9897be06 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:16.551835 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f41890-f03a-4730-9030-f37a9897be06 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:16.552115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f41890-f03a-4730-9030-f37a9897be06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:16.553278 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f41890-f03a-4730-9030-f37a9897be06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:16.554773 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f41890-f03a-4730-9030-f37a9897be06 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:16.556491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 101/220] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:16.557714 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f27237-e6cc-4797-934a-62bf8fba7aaf None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:16.558008 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f27237-e6cc-4797-934a-62bf8fba7aaf None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:16.558285 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f27237-e6cc-4797-934a-62bf8fba7aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:16.575826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c2f27237-e6cc-4797-934a-62bf8fba7aaf None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:16.577001 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c2f27237-e6cc-4797-934a-62bf8fba7aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:16.639662 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2cfb0-4c42-4a66-9415-cab7b95808ca None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:16.643163 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 102/221] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:16.643635 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f27237-e6cc-4797-934a-62bf8fba7aaf None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:16.674759 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6def4-faaa-4342-bc15-e42d4945dec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:16.675563 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6def4-faaa-4342-bc15-e42d4945dec8 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:16.675849 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6def4-faaa-4342-bc15-e42d4945dec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:16.676124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6def4-faaa-4342-bc15-e42d4945dec8 None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:16.677312 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6def4-faaa-4342-bc15-e42d4945dec8 None admin] RBAC: Authorizing `identity:get_project(project_id=b05339ac8c27421a9a1921726dcbdc7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:16.678801 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6def4-faaa-4342-bc15-e42d4945dec8 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:16.680641 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 120/222] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:16 2022] GET /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:16.681226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6def4-faaa-4342-bc15-e42d4945dec8 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:16.710553 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:16.741668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f27237-e6cc-4797-934a-62bf8fba7aaf None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:16.750363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:16.778555 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26eed10-04e5-4c0e-9406-7e817ced6adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:16.779295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26eed10-04e5-4c0e-9406-7e817ced6adb None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:16.779582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26eed10-04e5-4c0e-9406-7e817ced6adb None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:16.779865 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26eed10-04e5-4c0e-9406-7e817ced6adb None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:16.780613 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26eed10-04e5-4c0e-9406-7e817ced6adb None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:16.780978 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26eed10-04e5-4c0e-9406-7e817ced6adb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:16.782423 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26eed10-04e5-4c0e-9406-7e817ced6adb None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:16.785887 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6def4-faaa-4342-bc15-e42d4945dec8 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:16.790709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 103/223] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:16.791498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26eed10-04e5-4c0e-9406-7e817ced6adb None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:16.832489 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdac624-a8ea-4c75-8827-44c71eb7a530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:16.833158 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdac624-a8ea-4c75-8827-44c71eb7a530 None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:16.833442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdac624-a8ea-4c75-8827-44c71eb7a530 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:16.833716 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdac624-a8ea-4c75-8827-44c71eb7a530 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:16.834429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdac624-a8ea-4c75-8827-44c71eb7a530 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:16.835929 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdac624-a8ea-4c75-8827-44c71eb7a530 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:16.894092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26eed10-04e5-4c0e-9406-7e817ced6adb None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:16.914043 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 121/224] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 23 11:25:16 2022] POST /identity/v3/users => generated 297 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:25:16.915209 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdac624-a8ea-4c75-8827-44c71eb7a530 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:16.927942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b445cc-e577-4572-b675-a19154545dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:16.928788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b445cc-e577-4572-b675-a19154545dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:16.929077 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b445cc-e577-4572-b675-a19154545dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:16.929356 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b445cc-e577-4572-b675-a19154545dc1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:16.942684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b445cc-e577-4572-b675-a19154545dc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:16.951601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-52b445cc-e577-4572-b675-a19154545dc1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:16.951601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:16.951601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:16.951601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:16.951601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:16.951601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:16.951601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:16.951601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:16.951601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:16.951601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:16.951601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:16.951601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:16.951601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:16.951601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:16.951601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:16.951601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:16.952583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:16.952583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:16.952583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:16.952583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:16.952583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:16.952583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:16.952583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:16.952583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:16.952583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:16.952583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:16.952583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:16.952583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:16.952583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:16.952583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:16.952583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:16.952583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:16.953396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:16.953396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:16.953396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:16.953396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:16.953396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:16.953396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:16.953396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:16.953396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:16.953396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:16.953396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:16.953396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:16.953396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:16.953396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 104/225] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:25:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:16.954899 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b445cc-e577-4572-b675-a19154545dc1 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:16.989267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:17.043551 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:17.064472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdac624-a8ea-4c75-8827-44c71eb7a530 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:17.084872 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b445cc-e577-4572-b675-a19154545dc1 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:17.107932 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b43be-4e8f-4f10-b57f-06380b79f5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:17.109397 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b43be-4e8f-4f10-b57f-06380b79f5ef None admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:17.109802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b43be-4e8f-4f10-b57f-06380b79f5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:17.110086 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b43be-4e8f-4f10-b57f-06380b79f5ef None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f/users/159c476f43a84914883a4ee264db9ff9/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:17.112975 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3b43be-4e8f-4f10-b57f-06380b79f5ef None admin] RBAC: Authorizing `identity:create_grant(project_id=b05339ac8c27421a9a1921726dcbdc7f, user_id=159c476f43a84914883a4ee264db9ff9, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:17.113681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3b43be-4e8f-4f10-b57f-06380b79f5ef None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:17.128510 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d920d0-45cd-4bd1-9779-ecbf057c8eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:17.129606 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d920d0-45cd-4bd1-9779-ecbf057c8eea None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:17.129718 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d920d0-45cd-4bd1-9779-ecbf057c8eea None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:17.129933 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d920d0-45cd-4bd1-9779-ecbf057c8eea None admin] PATH_INFO: `/v3/users/7b06f25a47da4c4b80db567a34eae41c` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:17.145640 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d920d0-45cd-4bd1-9779-ecbf057c8eea None admin] RBAC: Authorizing `identity:get_user(user_id=7b06f25a47da4c4b80db567a34eae41c)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:17.147096 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d920d0-45cd-4bd1-9779-ecbf057c8eea None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:17.149139 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 105/226] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:16 2022] GET /identity/v3/users/7b06f25a47da4c4b80db567a34eae41c => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:17.157255 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d920d0-45cd-4bd1-9779-ecbf057c8eea None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:17.205202 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 122/227] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:25:16 2022] PUT /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f/users/159c476f43a84914883a4ee264db9ff9/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:25:17.206275 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b43be-4e8f-4f10-b57f-06380b79f5ef None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:17.256424 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:17.274314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d920d0-45cd-4bd1-9779-ecbf057c8eea None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:17.325666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af6b4c-4613-40b6-80c3-9a008fa2659d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:17.331471 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af6b4c-4613-40b6-80c3-9a008fa2659d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:17.332036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af6b4c-4613-40b6-80c3-9a008fa2659d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:17.332495 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af6b4c-4613-40b6-80c3-9a008fa2659d None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:17.333455 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af6b4c-4613-40b6-80c3-9a008fa2659d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:17.333973 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af6b4c-4613-40b6-80c3-9a008fa2659d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:17.335626 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af6b4c-4613-40b6-80c3-9a008fa2659d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:17.354044 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 106/228] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:25:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:17.363653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af6b4c-4613-40b6-80c3-9a008fa2659d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:17.382195 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b43be-4e8f-4f10-b57f-06380b79f5ef None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:17.433694 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207323c-f91d-4678-ac62-efb61f815169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:17.434876 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207323c-f91d-4678-ac62-efb61f815169 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:17.435697 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207323c-f91d-4678-ac62-efb61f815169 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:17.436179 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207323c-f91d-4678-ac62-efb61f815169 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:17.451735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:17.498795 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af6b4c-4613-40b6-80c3-9a008fa2659d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:17.558860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207323c-f91d-4678-ac62-efb61f815169 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:17.561201 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207323c-f91d-4678-ac62-efb61f815169 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:17.568538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee7440-2d03-4d95-a285-badba23b1ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:17.569458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee7440-2d03-4d95-a285-badba23b1ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:17.569739 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee7440-2d03-4d95-a285-badba23b1ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:17.570012 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee7440-2d03-4d95-a285-badba23b1ad1 None admin] PATH_INFO: `/v3/users/159c476f43a84914883a4ee264db9ff9` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:17.571286 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee7440-2d03-4d95-a285-badba23b1ad1 None admin] RBAC: Authorizing `identity:get_user(user_id=159c476f43a84914883a4ee264db9ff9)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:17.572728 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee7440-2d03-4d95-a285-badba23b1ad1 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:17.576149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 107/229] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:17 2022] GET /identity/v3/users/159c476f43a84914883a4ee264db9ff9 => generated 283 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:17.583878 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee7440-2d03-4d95-a285-badba23b1ad1 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:17.617781 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:17.716615 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 123/230] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:17.722879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee7440-2d03-4d95-a285-badba23b1ad1 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:17.727144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207323c-f91d-4678-ac62-efb61f815169 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:17.762849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6997b82-3b5a-4fd6-a598-f0fdfdac1b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:17.763719 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6997b82-3b5a-4fd6-a598-f0fdfdac1b05 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:17.764001 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6997b82-3b5a-4fd6-a598-f0fdfdac1b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:17.764274 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6997b82-3b5a-4fd6-a598-f0fdfdac1b05 None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:17.766711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6997b82-3b5a-4fd6-a598-f0fdfdac1b05 None admin] RBAC: Authorizing `identity:get_project(project_id=b05339ac8c27421a9a1921726dcbdc7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:17.772674 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6997b82-3b5a-4fd6-a598-f0fdfdac1b05 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:17.772674 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 108/231] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:17 2022] GET /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:17.775832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6997b82-3b5a-4fd6-a598-f0fdfdac1b05 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:17.807126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:17.892053 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:17.910560 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6997b82-3b5a-4fd6-a598-f0fdfdac1b05 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:17.911685 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207323c-f91d-4678-ac62-efb61f815169 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:17.947006 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ef5a1-e72a-4578-9b0b-4252a0c1cdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:17.947006 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ef5a1-e72a-4578-9b0b-4252a0c1cdf2 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:17.947006 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ef5a1-e72a-4578-9b0b-4252a0c1cdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:17.947006 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ef5a1-e72a-4578-9b0b-4252a0c1cdf2 None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f/users/159c476f43a84914883a4ee264db9ff9/roles/25e99b2a08914aa0ab18cc4955f25364` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:17.947456 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ef5a1-e72a-4578-9b0b-4252a0c1cdf2 None admin] RBAC: Authorizing `identity:create_grant(project_id=b05339ac8c27421a9a1921726dcbdc7f, user_id=159c476f43a84914883a4ee264db9ff9, role_id=25e99b2a08914aa0ab18cc4955f25364)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:17.948753 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ef5a1-e72a-4578-9b0b-4252a0c1cdf2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:17.975885 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03bfcd-693f-4b96-aed5-457585fdaa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:17.977133 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03bfcd-693f-4b96-aed5-457585fdaa9f None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:17.977643 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03bfcd-693f-4b96-aed5-457585fdaa9f None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:17.978098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03bfcd-693f-4b96-aed5-457585fdaa9f None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:17.979581 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03bfcd-693f-4b96-aed5-457585fdaa9f None admin] RBAC: Authorizing `identity:get_project(project_id=b05339ac8c27421a9a1921726dcbdc7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:17.981283 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03bfcd-693f-4b96-aed5-457585fdaa9f None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:17.984932 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 124/232] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:17 2022] GET /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:17.990314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03bfcd-693f-4b96-aed5-457585fdaa9f None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:18.056053 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 109/233] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:25:17 2022] PUT /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f/users/159c476f43a84914883a4ee264db9ff9/roles/25e99b2a08914aa0ab18cc4955f25364 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:25:18.114074 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03bfcd-693f-4b96-aed5-457585fdaa9f None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:18.124057 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:18.158268 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe27b7-5db1-4617-917e-fa320ce0f9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:18.158978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe27b7-5db1-4617-917e-fa320ce0f9aa None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:18.159271 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe27b7-5db1-4617-917e-fa320ce0f9aa None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:18.159549 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe27b7-5db1-4617-917e-fa320ce0f9aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:18.160402 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe27b7-5db1-4617-917e-fa320ce0f9aa None admin] RBAC: Adding query filter params (role.id=75f08cd0671f436c9f968976aa8baa7f, scope.project.id=b05339ac8c27421a9a1921726dcbdc7f, user.id=7b06f25a47da4c4b80db567a34eae41c) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:18.160717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe27b7-5db1-4617-917e-fa320ce0f9aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:18.162209 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe27b7-5db1-4617-917e-fa320ce0f9aa None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:18.174739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 125/234] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:25:17 2022] GET /identity/v3/role_assignments?user.id=7b06f25a47da4c4b80db567a34eae41c&scope.project.id=b05339ac8c27421a9a1921726dcbdc7f&role.id=75f08cd0671f436c9f968976aa8baa7f => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:19.622426 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada7656-c2a3-44a3-8db4-2b92beafec38 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:19.622426 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada7656-c2a3-44a3-8db4-2b92beafec38 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:19.622426 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada7656-c2a3-44a3-8db4-2b92beafec38 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:19.622426 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 110/235] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:19.623151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac1014-c93f-4abe-a47e-38b36ec0a350 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:19.623151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac1014-c93f-4abe-a47e-38b36ec0a350 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:19.623151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac1014-c93f-4abe-a47e-38b36ec0a350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:19.623151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-56ac1014-c93f-4abe-a47e-38b36ec0a350 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:19.623151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-56ac1014-c93f-4abe-a47e-38b36ec0a350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:19.623728 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018c489-d6cf-4a85-a842-dbb03587628a None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:19.623728 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018c489-d6cf-4a85-a842-dbb03587628a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:19.623728 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018c489-d6cf-4a85-a842-dbb03587628a None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:19.623728 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 111/236] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:19.623728 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24807ae-1652-418d-bd56-7fbc32371ce4 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:19.623728 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24807ae-1652-418d-bd56-7fbc32371ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:19.624089 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24807ae-1652-418d-bd56-7fbc32371ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:19.624089 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c24807ae-1652-418d-bd56-7fbc32371ce4 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:19.624089 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c24807ae-1652-418d-bd56-7fbc32371ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:19.633295 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 126/237] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:19.654823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac1014-c93f-4abe-a47e-38b36ec0a350 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:19.703884 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 112/238] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:19.736522 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24807ae-1652-418d-bd56-7fbc32371ce4 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:19.803967 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac1014-c93f-4abe-a47e-38b36ec0a350 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:19.805497 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:19.816925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:19.842463 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24807ae-1652-418d-bd56-7fbc32371ce4 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:19.843984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3aa62-0467-4377-990e-1492e2199945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:19.845060 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3aa62-0467-4377-990e-1492e2199945 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:19.845552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3aa62-0467-4377-990e-1492e2199945 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:19.846025 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3aa62-0467-4377-990e-1492e2199945 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:19.857145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3aa62-0467-4377-990e-1492e2199945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:19.865477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-b6e3aa62-0467-4377-990e-1492e2199945 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:25:19.865477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:19.865477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:19.865477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:19.865477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:19.865477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:19.865477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:19.865477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:19.865477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:19.865477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:19.865477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:19.865477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:19.865477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:19.865477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:19.865477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:19.865477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:19.866401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:19.866401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:19.866401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:19.866401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:19.866401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:19.866401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:19.866401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:19.866401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:19.866401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:19.866401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:19.866401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:19.866401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:19.866401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:19.866401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:19.866401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:19.866401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:19.867514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:19.867514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:19.867514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:19.867514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:19.867514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:19.867514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:19.867514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:19.867514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:19.867514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:19.867514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:19.867514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:25:19.867514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:19.870241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 127/239] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:25:19.878840 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba1948-341d-402c-b949-969e447ea5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:19.879597 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3aa62-0467-4377-990e-1492e2199945 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:19.880015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba1948-341d-402c-b949-969e447ea5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:19.880718 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba1948-341d-402c-b949-969e447ea5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:19.881927 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba1948-341d-402c-b949-969e447ea5b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:19.883436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba1948-341d-402c-b949-969e447ea5b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:19.885013 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba1948-341d-402c-b949-969e447ea5b8 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:19.887136 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 113/240] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:19.893265 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba1948-341d-402c-b949-969e447ea5b8 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:19.996843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba1948-341d-402c-b949-969e447ea5b8 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:19.998724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3aa62-0467-4377-990e-1492e2199945 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:20.034867 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88ca0c-9077-4d77-931d-645ab11cfcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:20.035183 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f96ea-d581-410f-bb1d-25bf426d5edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:20.035550 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88ca0c-9077-4d77-931d-645ab11cfcf2 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:20.035833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88ca0c-9077-4d77-931d-645ab11cfcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:20.036105 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88ca0c-9077-4d77-931d-645ab11cfcf2 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:20.036465 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f96ea-d581-410f-bb1d-25bf426d5edd None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:20.036826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88ca0c-9077-4d77-931d-645ab11cfcf2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:20.037030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f96ea-d581-410f-bb1d-25bf426d5edd None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:20.037454 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f96ea-d581-410f-bb1d-25bf426d5edd None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:20.038281 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88ca0c-9077-4d77-931d-645ab11cfcf2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:20.038469 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f96ea-d581-410f-bb1d-25bf426d5edd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:20.038929 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f96ea-d581-410f-bb1d-25bf426d5edd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:20.040428 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f96ea-d581-410f-bb1d-25bf426d5edd None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:20.050255 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 128/241] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:20.051702 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbab48-5ee2-472c-9b0c-7699c6646297 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:20.051991 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbab48-5ee2-472c-9b0c-7699c6646297 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:20.052267 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbab48-5ee2-472c-9b0c-7699c6646297 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:20.053543 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 129/242] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:20.058970 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206232f4-9516-470c-9d7a-35ec6256f285 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:20.059255 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206232f4-9516-470c-9d7a-35ec6256f285 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:20.059525 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206232f4-9516-470c-9d7a-35ec6256f285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:20.077495 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-206232f4-9516-470c-9d7a-35ec6256f285 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:20.079001 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-206232f4-9516-470c-9d7a-35ec6256f285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:20.102813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:20.139339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:20.145731 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 114/243] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 23 11:25:19 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:25:20.146972 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88ca0c-9077-4d77-931d-645ab11cfcf2 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:20.193144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 130/244] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:20.227939 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206232f4-9516-470c-9d7a-35ec6256f285 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:20.350650 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88ca0c-9077-4d77-931d-645ab11cfcf2 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:20.390582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d7707-fc33-4748-ab27-4c6a56ff0556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:20.392096 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d7707-fc33-4748-ab27-4c6a56ff0556 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:20.392378 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d7707-fc33-4748-ab27-4c6a56ff0556 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:20.392652 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d7707-fc33-4748-ab27-4c6a56ff0556 None admin] PATH_INFO: `/v3/users/7e4a75e268104e27a0feae94e065971e` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:20.393258 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206232f4-9516-470c-9d7a-35ec6256f285 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:20.408316 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d7707-fc33-4748-ab27-4c6a56ff0556 None admin] RBAC: Authorizing `identity:get_user(user_id=7e4a75e268104e27a0feae94e065971e)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:20.409715 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d7707-fc33-4748-ab27-4c6a56ff0556 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:20.412116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 115/245] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:20 2022] GET /identity/v3/users/7e4a75e268104e27a0feae94e065971e => generated 297 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:20.417501 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d7707-fc33-4748-ab27-4c6a56ff0556 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:20.430977 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c357853-d119-4d8d-ab20-8bda7b3f4b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:20.432123 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c357853-d119-4d8d-ab20-8bda7b3f4b9d None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:20.432613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c357853-d119-4d8d-ab20-8bda7b3f4b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:20.433109 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c357853-d119-4d8d-ab20-8bda7b3f4b9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:20.445886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c357853-d119-4d8d-ab20-8bda7b3f4b9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:20.455894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-1c357853-d119-4d8d-ab20-8bda7b3f4b9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:25:20.455894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:20.455894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:20.455894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:20.455894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:20.455894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:20.455894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:20.455894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:20.455894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:20.455894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:20.455894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:20.455894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:20.455894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:20.455894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:20.455894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:20.455894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:20.456807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:20.456807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:20.456807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:20.456807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:20.456807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:20.456807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:20.456807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:20.456807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:20.456807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:20.456807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:20.456807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:20.456807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:20.456807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:20.456807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:20.456807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:20.456807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:20.457692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:20.457692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:20.457692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:20.457692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:20.457692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:20.457692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:20.457692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:20.457692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:20.457692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:20.457692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:20.457692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:25:20.457692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:20.460218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 131/246] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:25:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:20.471563 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c357853-d119-4d8d-ab20-8bda7b3f4b9d None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:20.481563 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:20.500672 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:20.542643 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d7707-fc33-4748-ab27-4c6a56ff0556 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:20.594678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd9e64-7f10-4300-9d80-d13fc530fbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:20.595476 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd9e64-7f10-4300-9d80-d13fc530fbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:20.595769 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd9e64-7f10-4300-9d80-d13fc530fbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:20.596052 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd9e64-7f10-4300-9d80-d13fc530fbf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:20.629592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c357853-d119-4d8d-ab20-8bda7b3f4b9d None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:20.670337 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb1d84-1d01-4e81-bce7-2594e2ba8138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:20.670943 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb1d84-1d01-4e81-bce7-2594e2ba8138 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:20.671765 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb1d84-1d01-4e81-bce7-2594e2ba8138 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:20.672034 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb1d84-1d01-4e81-bce7-2594e2ba8138 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:20.672900 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb1d84-1d01-4e81-bce7-2594e2ba8138 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:20.673217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb1d84-1d01-4e81-bce7-2594e2ba8138 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:20.674679 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb1d84-1d01-4e81-bce7-2594e2ba8138 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:20.685971 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 132/247] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:20.695369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb1d84-1d01-4e81-bce7-2594e2ba8138 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:20.700879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddd9e64-7f10-4300-9d80-d13fc530fbf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:20.703572 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddd9e64-7f10-4300-9d80-d13fc530fbf9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:20.776222 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:20.792567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:20.816397 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb1d84-1d01-4e81-bce7-2594e2ba8138 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:20.829936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 116/248] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:20.831667 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f950e-2ccd-47d0-b1f4-b1e3df2e6210 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:20.832149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f950e-2ccd-47d0-b1f4-b1e3df2e6210 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:20.832595 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f950e-2ccd-47d0-b1f4-b1e3df2e6210 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:20.834343 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 117/249] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:20.835641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31ee77-bfa6-43d6-a47e-d1b3bed724c5 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:20.836125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31ee77-bfa6-43d6-a47e-d1b3bed724c5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:20.836570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31ee77-bfa6-43d6-a47e-d1b3bed724c5 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:20.837981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 118/250] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:20.839130 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31ee77-bfa6-43d6-a47e-d1b3bed724c5 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:20.858268 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a7406-a1f0-4da7-9345-36c21e675aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:20.859983 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a7406-a1f0-4da7-9345-36c21e675aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:20.860470 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a7406-a1f0-4da7-9345-36c21e675aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:20.860820 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a7406-a1f0-4da7-9345-36c21e675aa6 None admin] PATH_INFO: `/v3/users/159c476f43a84914883a4ee264db9ff9` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:20.862423 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a7406-a1f0-4da7-9345-36c21e675aa6 None admin] RBAC: Authorizing `identity:get_user(user_id=159c476f43a84914883a4ee264db9ff9)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:20.863974 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a7406-a1f0-4da7-9345-36c21e675aa6 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:20.866861 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 133/251] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:20 2022] GET /identity/v3/users/159c476f43a84914883a4ee264db9ff9 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:20.868364 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122dbb9-41d6-4f83-b96f-970ef1e9f797 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:20.869046 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122dbb9-41d6-4f83-b96f-970ef1e9f797 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:20.869402 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122dbb9-41d6-4f83-b96f-970ef1e9f797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:20.890653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-b122dbb9-41d6-4f83-b96f-970ef1e9f797 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:20.892457 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b122dbb9-41d6-4f83-b96f-970ef1e9f797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:20.960729 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31ee77-bfa6-43d6-a47e-d1b3bed724c5 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:20.986619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 134/252] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:20.988347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf146b1-a706-4639-b562-512ce063d101 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:20.988854 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf146b1-a706-4639-b562-512ce063d101 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:20.989317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf146b1-a706-4639-b562-512ce063d101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:21.014792 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0741de-dd15-42b8-bafb-c132013ea592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:21.015860 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0741de-dd15-42b8-bafb-c132013ea592 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:21.016142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0741de-dd15-42b8-bafb-c132013ea592 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:21.016415 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0741de-dd15-42b8-bafb-c132013ea592 None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:21.017747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0741de-dd15-42b8-bafb-c132013ea592 None admin] RBAC: Authorizing `identity:get_project(project_id=b05339ac8c27421a9a1921726dcbdc7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:21.019232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0741de-dd15-42b8-bafb-c132013ea592 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:21.021148 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 119/253] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:20 2022] GET /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:21.021700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0741de-dd15-42b8-bafb-c132013ea592 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:21.025718 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-ecf146b1-a706-4639-b562-512ce063d101 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:21.027190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-ecf146b1-a706-4639-b562-512ce063d101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:21.058097 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:21.100459 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 135/254] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:21.101105 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf146b1-a706-4639-b562-512ce063d101 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:21.104368 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:21.154377 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0741de-dd15-42b8-bafb-c132013ea592 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:21.201235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65a030-9bb5-4111-9068-b6dca60d70d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:21.202696 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65a030-9bb5-4111-9068-b6dca60d70d3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:21.203297 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65a030-9bb5-4111-9068-b6dca60d70d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:21.204055 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65a030-9bb5-4111-9068-b6dca60d70d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:21.226688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf146b1-a706-4639-b562-512ce063d101 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:21.266392 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65fa56-ab8d-45a2-b1e7-dc535b095d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:21.268051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65fa56-ab8d-45a2-b1e7-dc535b095d2d None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:21.268762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65fa56-ab8d-45a2-b1e7-dc535b095d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:21.269202 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65fa56-ab8d-45a2-b1e7-dc535b095d2d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:21.284972 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a65fa56-ab8d-45a2-b1e7-dc535b095d2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:21.290152 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65a030-9bb5-4111-9068-b6dca60d70d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:21.291756 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65a030-9bb5-4111-9068-b6dca60d70d3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:21.295834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-6a65fa56-ab8d-45a2-b1e7-dc535b095d2d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:21.295834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:21.295834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:21.295834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:21.295834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:21.295834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:21.295834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:21.295834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:21.295834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:21.295834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:21.295834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:21.295834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:21.295834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:21.295834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:21.295834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:21.295834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:21.297632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:21.297632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:21.297632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:21.297632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:21.297632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:21.297632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:21.297632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:21.297632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:21.297632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:21.297632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:21.297632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:21.297632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:21.297632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:21.297632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:21.297632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:21.297632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:21.298620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:21.298620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:21.298620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:21.298620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:21.298620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:21.298620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:21.298620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:21.298620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:21.298620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:21.298620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:21.298620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:21.298620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:21.300687 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 136/255] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:21.301602 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65fa56-ab8d-45a2-b1e7-dc535b095d2d None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:21.321147 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:21.388688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:21.406481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 120/256] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:21.411585 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65a030-9bb5-4111-9068-b6dca60d70d3 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:21.431608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65fa56-ab8d-45a2-b1e7-dc535b095d2d None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:21.467981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31979325-25de-473d-8256-cd8ca211fcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:21.468959 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31979325-25de-473d-8256-cd8ca211fcf2 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:21.469417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31979325-25de-473d-8256-cd8ca211fcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:21.469867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31979325-25de-473d-8256-cd8ca211fcf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:21.470929 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31979325-25de-473d-8256-cd8ca211fcf2 None admin] RBAC: Adding query filter params (role.id=c8e4f6d521b04ad39661b74575a8b9d3, scope.project.id=b05339ac8c27421a9a1921726dcbdc7f, user.id=7e4a75e268104e27a0feae94e065971e) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:21.471462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31979325-25de-473d-8256-cd8ca211fcf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:21.473113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31979325-25de-473d-8256-cd8ca211fcf2 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:21.486676 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 137/257] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:25:21 2022] GET /identity/v3/role_assignments?user.id=7e4a75e268104e27a0feae94e065971e&scope.project.id=b05339ac8c27421a9a1921726dcbdc7f&role.id=c8e4f6d521b04ad39661b74575a8b9d3 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:21.488266 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31979325-25de-473d-8256-cd8ca211fcf2 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:21.527980 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65a030-9bb5-4111-9068-b6dca60d70d3 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:21.556317 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:21.571670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e239355e-482b-4e63-b01a-23354ce8268c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:21.572555 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239355e-482b-4e63-b01a-23354ce8268c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:21.572840 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239355e-482b-4e63-b01a-23354ce8268c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:21.573160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239355e-482b-4e63-b01a-23354ce8268c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:21.597748 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e239355e-482b-4e63-b01a-23354ce8268c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:21.612068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-e239355e-482b-4e63-b01a-23354ce8268c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:21.612068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:21.612068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:21.612068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:21.612068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:21.612068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:21.612068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:21.612068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:21.612068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:21.612068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:21.612068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:21.612068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:21.612068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:21.612068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:21.612068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:21.612068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:21.613027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:21.613027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:21.613027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:21.613027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:21.613027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:21.613027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:21.613027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:21.613027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:21.613027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:21.613027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:21.613027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:21.613027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:21.613027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:21.613027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:21.613027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:21.613027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:21.613814 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:21.613814 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:21.613814 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:21.613814 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:21.613814 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:21.613814 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:21.613814 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:21.613814 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:21.613814 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:21.613814 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:21.613814 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:21.613814 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:21.615327 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 121/258] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:25:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:21.618439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e239355e-482b-4e63-b01a-23354ce8268c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:21.686568 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31979325-25de-473d-8256-cd8ca211fcf2 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:21.721391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:21.733299 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ad749-8919-4250-a0d6-257a269e550f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:21.735329 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ad749-8919-4250-a0d6-257a269e550f None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:21.736096 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ad749-8919-4250-a0d6-257a269e550f None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:21.737029 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ad749-8919-4250-a0d6-257a269e550f None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:21.738273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ad749-8919-4250-a0d6-257a269e550f None admin] RBAC: Adding query filter params (name=member) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:21.739068 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ad749-8919-4250-a0d6-257a269e550f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:21.741376 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ad749-8919-4250-a0d6-257a269e550f None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:21.755920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 138/259] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:21.757149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ad749-8919-4250-a0d6-257a269e550f None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:21.826312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e239355e-482b-4e63-b01a-23354ce8268c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:21.877171 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ad749-8919-4250-a0d6-257a269e550f None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:21.917382 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186ae93-58e8-468e-a618-c6f9e1669555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:21.918590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186ae93-58e8-468e-a618-c6f9e1669555 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:21.919091 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186ae93-58e8-468e-a618-c6f9e1669555 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:21.919575 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186ae93-58e8-468e-a618-c6f9e1669555 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:21.920517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186ae93-58e8-468e-a618-c6f9e1669555 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:21.921038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186ae93-58e8-468e-a618-c6f9e1669555 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:21.922896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186ae93-58e8-468e-a618-c6f9e1669555 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:21.932646 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 139/260] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:25:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:21.934056 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186ae93-58e8-468e-a618-c6f9e1669555 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:21.947145 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0397398-50bb-40dc-b011-608fa6984b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:21.947940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0397398-50bb-40dc-b011-608fa6984b3f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:21.948224 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0397398-50bb-40dc-b011-608fa6984b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:21.948501 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0397398-50bb-40dc-b011-608fa6984b3f None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:21.949702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0397398-50bb-40dc-b011-608fa6984b3f None admin] RBAC: Authorizing `identity:get_project(project_id=b05339ac8c27421a9a1921726dcbdc7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:21.951920 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0397398-50bb-40dc-b011-608fa6984b3f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:21.960058 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 122/261] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:21 2022] GET /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f => generated 302 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:21.960848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0397398-50bb-40dc-b011-608fa6984b3f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:22.002800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:22.055648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186ae93-58e8-468e-a618-c6f9e1669555 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:22.093286 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00e0cc-c51d-4439-a105-d29fe1672d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:22.094512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00e0cc-c51d-4439-a105-d29fe1672d89 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:22.095337 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00e0cc-c51d-4439-a105-d29fe1672d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:22.095813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00e0cc-c51d-4439-a105-d29fe1672d89 None admin] PATH_INFO: `/v3/users/7b06f25a47da4c4b80db567a34eae41c` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:22.097217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00e0cc-c51d-4439-a105-d29fe1672d89 None admin] RBAC: Authorizing `identity:get_user(user_id=7b06f25a47da4c4b80db567a34eae41c)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:22.099256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00e0cc-c51d-4439-a105-d29fe1672d89 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:22.101393 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 140/262] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:21 2022] GET /identity/v3/users/7b06f25a47da4c4b80db567a34eae41c => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:22.102366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00e0cc-c51d-4439-a105-d29fe1672d89 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:22.104320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:22.212091 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00e0cc-c51d-4439-a105-d29fe1672d89 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:22.234892 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:22.251153 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f9923-90e5-4841-a7fb-988a56c55f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:22.252113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f9923-90e5-4841-a7fb-988a56c55f4a None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:22.252623 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f9923-90e5-4841-a7fb-988a56c55f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:22.253100 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f9923-90e5-4841-a7fb-988a56c55f4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:22.254127 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f9923-90e5-4841-a7fb-988a56c55f4a None admin] RBAC: Adding query filter params (role.id=c9e006de0d6c4812ab29d778e9c8a830, scope.project.id=b05339ac8c27421a9a1921726dcbdc7f, user.id=159c476f43a84914883a4ee264db9ff9) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:22.254813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f9923-90e5-4841-a7fb-988a56c55f4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:22.256472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f9923-90e5-4841-a7fb-988a56c55f4a None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:22.263060 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 141/263] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:25:22 2022] GET /identity/v3/role_assignments?user.id=159c476f43a84914883a4ee264db9ff9&scope.project.id=b05339ac8c27421a9a1921726dcbdc7f&role.id=c9e006de0d6c4812ab29d778e9c8a830 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:22.264647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f9923-90e5-4841-a7fb-988a56c55f4a None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:22.293742 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0397398-50bb-40dc-b011-608fa6984b3f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:22.377427 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f9923-90e5-4841-a7fb-988a56c55f4a None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:22.383455 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf916028-4aeb-4e8d-a922-ba805ebfe176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:22.384381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf916028-4aeb-4e8d-a922-ba805ebfe176 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:22.384668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf916028-4aeb-4e8d-a922-ba805ebfe176 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:22.384949 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf916028-4aeb-4e8d-a922-ba805ebfe176 None admin] PATH_INFO: `/v3/users/159c476f43a84914883a4ee264db9ff9` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:22.386334 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf916028-4aeb-4e8d-a922-ba805ebfe176 None admin] RBAC: Authorizing `identity:get_user(user_id=159c476f43a84914883a4ee264db9ff9)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:22.387870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf916028-4aeb-4e8d-a922-ba805ebfe176 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:22.390459 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 123/264] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:21 2022] GET /identity/v3/users/159c476f43a84914883a4ee264db9ff9 => generated 283 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:22.399682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf916028-4aeb-4e8d-a922-ba805ebfe176 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:22.456698 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d2367-0713-4d78-83ea-0e5be5a62838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:22.457972 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d2367-0713-4d78-83ea-0e5be5a62838 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:22.458517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d2367-0713-4d78-83ea-0e5be5a62838 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:22.459047 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d2367-0713-4d78-83ea-0e5be5a62838 None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:22.463564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d2367-0713-4d78-83ea-0e5be5a62838 None admin] RBAC: Authorizing `identity:get_project(project_id=b05339ac8c27421a9a1921726dcbdc7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:22.465000 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d2367-0713-4d78-83ea-0e5be5a62838 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:22.470613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 142/265] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:22 2022] GET /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:22.476629 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d2367-0713-4d78-83ea-0e5be5a62838 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:22.576531 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:22.588065 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf916028-4aeb-4e8d-a922-ba805ebfe176 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:22.631048 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:22.645741 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb8fa1-ddc0-4866-89f4-e6fa3ee84edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:22.646405 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb8fa1-ddc0-4866-89f4-e6fa3ee84edd None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:22.646722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb8fa1-ddc0-4866-89f4-e6fa3ee84edd None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:22.647001 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb8fa1-ddc0-4866-89f4-e6fa3ee84edd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:22.662820 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d2367-0713-4d78-83ea-0e5be5a62838 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:22.724244 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb8fa1-ddc0-4866-89f4-e6fa3ee84edd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:22.725657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb8fa1-ddc0-4866-89f4-e6fa3ee84edd None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:22.727102 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a37d1-fdd5-4ee1-8e2a-d481d65f29be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:22.728706 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a37d1-fdd5-4ee1-8e2a-d481d65f29be None admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:22.729128 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a37d1-fdd5-4ee1-8e2a-d481d65f29be None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:22.729540 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a37d1-fdd5-4ee1-8e2a-d481d65f29be None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f/users/7b06f25a47da4c4b80db567a34eae41c/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:22.735653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4a37d1-fdd5-4ee1-8e2a-d481d65f29be None admin] RBAC: Authorizing `identity:create_grant(project_id=b05339ac8c27421a9a1921726dcbdc7f, user_id=7b06f25a47da4c4b80db567a34eae41c, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:22.737425 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4a37d1-fdd5-4ee1-8e2a-d481d65f29be None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:22.814508 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 143/266] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:25:22 2022] PUT /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f/users/7b06f25a47da4c4b80db567a34eae41c/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:25:22.835278 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 124/267] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:22.840617 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a37d1-fdd5-4ee1-8e2a-d481d65f29be None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:22.935266 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777597c-ce8d-4c7c-81a9-c1a6bdc94259 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:22.935581 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777597c-ce8d-4c7c-81a9-c1a6bdc94259 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:22.935836 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777597c-ce8d-4c7c-81a9-c1a6bdc94259 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:22.938049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 125/268] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:22.943776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec3634-0244-4125-8be6-fa69c2b2fef0 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:22.944089 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec3634-0244-4125-8be6-fa69c2b2fef0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:22.944334 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec3634-0244-4125-8be6-fa69c2b2fef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:22.962227 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-6cec3634-0244-4125-8be6-fa69c2b2fef0 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:22.963468 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6cec3634-0244-4125-8be6-fa69c2b2fef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:22.968960 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a37d1-fdd5-4ee1-8e2a-d481d65f29be None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:22.984283 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:22.986931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:23.007082 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1fb7c-d5c4-41be-aad7-421b4e1ca13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:23.007967 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1fb7c-d5c4-41be-aad7-421b4e1ca13c None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:23.008256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1fb7c-d5c4-41be-aad7-421b4e1ca13c None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:23.008536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1fb7c-d5c4-41be-aad7-421b4e1ca13c None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:23.010846 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1fb7c-d5c4-41be-aad7-421b4e1ca13c None admin] RBAC: Authorizing `identity:get_project(project_id=b05339ac8c27421a9a1921726dcbdc7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:23.012429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1fb7c-d5c4-41be-aad7-421b4e1ca13c None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:23.014522 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 144/269] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:22 2022] GET /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:23.020717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1fb7c-d5c4-41be-aad7-421b4e1ca13c None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:23.056959 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 126/270] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:23.101901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec3634-0244-4125-8be6-fa69c2b2fef0 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:23.168298 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1fb7c-d5c4-41be-aad7-421b4e1ca13c None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:23.214024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba88c9-de9f-48c3-b8e6-5f45e7f9149b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:23.215024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba88c9-de9f-48c3-b8e6-5f45e7f9149b None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:23.215323 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba88c9-de9f-48c3-b8e6-5f45e7f9149b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:23.215604 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba88c9-de9f-48c3-b8e6-5f45e7f9149b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:23.216724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ba88c9-de9f-48c3-b8e6-5f45e7f9149b None admin] RBAC: Adding query filter params (role.id=25e99b2a08914aa0ab18cc4955f25364, scope.project.id=b05339ac8c27421a9a1921726dcbdc7f, user.id=159c476f43a84914883a4ee264db9ff9) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:23.217041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ba88c9-de9f-48c3-b8e6-5f45e7f9149b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:23.218803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ba88c9-de9f-48c3-b8e6-5f45e7f9149b None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:23.223000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec3634-0244-4125-8be6-fa69c2b2fef0 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:23.226839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 145/271] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:25:23 2022] GET /identity/v3/role_assignments?user.id=159c476f43a84914883a4ee264db9ff9&scope.project.id=b05339ac8c27421a9a1921726dcbdc7f&role.id=25e99b2a08914aa0ab18cc4955f25364 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:23.228134 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096f891-cdbc-40fb-bf8b-ee203299ed18 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:23.228429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096f891-cdbc-40fb-bf8b-ee203299ed18 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:23.228711 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096f891-cdbc-40fb-bf8b-ee203299ed18 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:23.230171 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 146/272] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:23.236793 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cffd24-c0bf-48ee-b189-472894d4f779 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:23.237084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cffd24-c0bf-48ee-b189-472894d4f779 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:23.237278 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cffd24-c0bf-48ee-b189-472894d4f779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:23.259348 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-84cffd24-c0bf-48ee-b189-472894d4f779 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:23.261227 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-84cffd24-c0bf-48ee-b189-472894d4f779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:23.271327 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6a897-68c6-4f01-8a10-8a94d6eac4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:23.272492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6a897-68c6-4f01-8a10-8a94d6eac4cc None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:23.273417 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6a897-68c6-4f01-8a10-8a94d6eac4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:23.273894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6a897-68c6-4f01-8a10-8a94d6eac4cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:23.296303 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd6a897-68c6-4f01-8a10-8a94d6eac4cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:23.311245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-bbd6a897-68c6-4f01-8a10-8a94d6eac4cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:23.311245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:23.311245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:23.311245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:23.311245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:23.311245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:23.311245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:23.311245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:23.311245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:23.311245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:23.311245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:23.311245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:23.311245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:23.311245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:23.311245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:23.311245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:23.312764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:23.312764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:23.312764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:23.312764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:23.312764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:23.312764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:23.312764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:23.312764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:23.312764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:23.312764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:23.312764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:23.312764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:23.312764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:23.312764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:23.312764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:23.312764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:23.313671 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:23.313671 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:23.313671 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:23.313671 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:23.313671 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:23.313671 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:23.313671 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:23.313671 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:23.313671 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:23.313671 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:23.313671 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:23.313671 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:23.330920 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 127/273] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:23.334614 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6a897-68c6-4f01-8a10-8a94d6eac4cc None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:23.342521 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:23.367462 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:23.377216 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 147/274] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:23.415144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cffd24-c0bf-48ee-b189-472894d4f779 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:23.552297 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6a897-68c6-4f01-8a10-8a94d6eac4cc None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:23.635757 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56fbdf-6c49-4c13-b348-ff6c8f12e3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:23.637511 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56fbdf-6c49-4c13-b348-ff6c8f12e3a0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:23.646553 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cffd24-c0bf-48ee-b189-472894d4f779 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:23.646708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56fbdf-6c49-4c13-b348-ff6c8f12e3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:23.646708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56fbdf-6c49-4c13-b348-ff6c8f12e3a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:23.646708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56fbdf-6c49-4c13-b348-ff6c8f12e3a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:23.646708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56fbdf-6c49-4c13-b348-ff6c8f12e3a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:23.646708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56fbdf-6c49-4c13-b348-ff6c8f12e3a0 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:23.670656 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 128/275] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:23.694088 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56fbdf-6c49-4c13-b348-ff6c8f12e3a0 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:23.717732 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6322f7-b84f-4377-93a8-51300d63dd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:23.720507 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6322f7-b84f-4377-93a8-51300d63dd3a None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:23.720507 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6322f7-b84f-4377-93a8-51300d63dd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:23.720507 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6322f7-b84f-4377-93a8-51300d63dd3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:23.724598 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6322f7-b84f-4377-93a8-51300d63dd3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:23.726044 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6322f7-b84f-4377-93a8-51300d63dd3a None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:23.732331 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 148/276] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:23.739482 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6322f7-b84f-4377-93a8-51300d63dd3a None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:23.843991 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:23.865342 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:23.902038 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56fbdf-6c49-4c13-b348-ff6c8f12e3a0 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:23.980468 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32f24c-c890-4280-9181-77196dddcaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:23.982254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32f24c-c890-4280-9181-77196dddcaad None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:23.982860 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32f24c-c890-4280-9181-77196dddcaad None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:23.985869 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32f24c-c890-4280-9181-77196dddcaad None admin] PATH_INFO: `/v3/users/2cda5981dcd54219b7d697b973badf36` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:24.022564 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32f24c-c890-4280-9181-77196dddcaad None admin] RBAC: Authorizing `identity:get_user(user_id=2cda5981dcd54219b7d697b973badf36)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:24.022564 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32f24c-c890-4280-9181-77196dddcaad None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:24.022564 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 129/277] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:23 2022] GET /identity/v3/users/2cda5981dcd54219b7d697b973badf36 => generated 293 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:24.028790 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32f24c-c890-4280-9181-77196dddcaad None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:24.049720 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6322f7-b84f-4377-93a8-51300d63dd3a None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:24.115410 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b38761-6232-456b-9d5d-1097ec39ff4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:24.116022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b38761-6232-456b-9d5d-1097ec39ff4e None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:24.116354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b38761-6232-456b-9d5d-1097ec39ff4e None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:24.116641 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b38761-6232-456b-9d5d-1097ec39ff4e None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:24.117634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b38761-6232-456b-9d5d-1097ec39ff4e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:24.119530 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b38761-6232-456b-9d5d-1097ec39ff4e None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:24.185077 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32f24c-c890-4280-9181-77196dddcaad None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:24.213445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:24.220034 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643430b2-7cb6-4c60-850a-6fff772bd02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:24.220688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643430b2-7cb6-4c60-850a-6fff772bd02d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:24.220969 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643430b2-7cb6-4c60-850a-6fff772bd02d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:24.221255 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643430b2-7cb6-4c60-850a-6fff772bd02d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:24.222145 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643430b2-7cb6-4c60-850a-6fff772bd02d None admin] RBAC: Adding query filter params (role.id=75f08cd0671f436c9f968976aa8baa7f, user.id=2cda5981dcd54219b7d697b973badf36, scope.system=all) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:24.222399 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643430b2-7cb6-4c60-850a-6fff772bd02d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:24.223870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643430b2-7cb6-4c60-850a-6fff772bd02d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:24.228727 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 130/278] 10.222.0.13 () {56 vars in 1308 bytes} [Thu Jun 23 11:25:24 2022] GET /identity/v3/role_assignments?user.id=2cda5981dcd54219b7d697b973badf36&scope.system=all&role.id=75f08cd0671f436c9f968976aa8baa7f => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:24.230062 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabed39-dcb6-4fa9-ba96-656edb067ab3 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:24.230354 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabed39-dcb6-4fa9-ba96-656edb067ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:24.230655 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabed39-dcb6-4fa9-ba96-656edb067ab3 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:24.232092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 131/279] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:24.238847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0e38d-7d25-4672-805c-59d5dbe6bd39 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:24.239142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0e38d-7d25-4672-805c-59d5dbe6bd39 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:24.239546 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0e38d-7d25-4672-805c-59d5dbe6bd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:24.256990 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b7a0e38d-7d25-4672-805c-59d5dbe6bd39 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:24.257450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 149/280] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 23 11:25:23 2022] POST /identity/v3/users => generated 293 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:25:24.258246 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b7a0e38d-7d25-4672-805c-59d5dbe6bd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:24.352914 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 132/281] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:25:24.373579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b38761-6232-456b-9d5d-1097ec39ff4e None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:24.494792 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:24.498385 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b38761-6232-456b-9d5d-1097ec39ff4e None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:24.535697 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3914c-b83c-4aff-ac48-ea51717884ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:24.536511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3914c-b83c-4aff-ac48-ea51717884ac None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:24.536795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3914c-b83c-4aff-ac48-ea51717884ac None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:24.537170 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3914c-b83c-4aff-ac48-ea51717884ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:24.548982 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3914c-b83c-4aff-ac48-ea51717884ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:24.556714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-48e3914c-b83c-4aff-ac48-ea51717884ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:25:24.556714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:24.556714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:24.556714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:24.556714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:24.556714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:24.556714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:24.556714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:24.556714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:24.556714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:24.556714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:24.556714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:24.556714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:24.556714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:24.556714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:24.556714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:24.557650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:24.557650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:24.557650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:24.557650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:24.557650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:24.557650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:24.557650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:24.557650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:24.557650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:24.557650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:24.557650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:24.557650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:24.557650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:24.557650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:24.557650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:24.557650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:24.558446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:24.558446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:24.558446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:24.558446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:24.558446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:24.558446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:24.558446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:24.558446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:24.558446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:24.558446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:24.558446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:25:24.558446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:24.561734 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 150/282] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:24.564570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0e38d-7d25-4672-805c-59d5dbe6bd39 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:24.673672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0e38d-7d25-4672-805c-59d5dbe6bd39 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:24.699098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:24.710947 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942db900-1c4e-487d-942f-f1ed1b080091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:24.712267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942db900-1c4e-487d-942f-f1ed1b080091 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:24.712267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942db900-1c4e-487d-942f-f1ed1b080091 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:24.712448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942db900-1c4e-487d-942f-f1ed1b080091 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:24.713212 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942db900-1c4e-487d-942f-f1ed1b080091 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:24.713519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942db900-1c4e-487d-942f-f1ed1b080091 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:24.714972 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942db900-1c4e-487d-942f-f1ed1b080091 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:24.724770 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 133/283] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:24.730377 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3914c-b83c-4aff-ac48-ea51717884ac None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:24.858325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3914c-b83c-4aff-ac48-ea51717884ac None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:24.898235 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b70887-7da6-427b-b3dd-eddf653e493e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:24.899150 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b70887-7da6-427b-b3dd-eddf653e493e None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:24.899436 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b70887-7da6-427b-b3dd-eddf653e493e None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:24.899713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b70887-7da6-427b-b3dd-eddf653e493e None admin] PATH_INFO: `/v3/users/7e4a75e268104e27a0feae94e065971e` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:24.900945 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b70887-7da6-427b-b3dd-eddf653e493e None admin] RBAC: Authorizing `identity:get_user(user_id=7e4a75e268104e27a0feae94e065971e)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:24.902395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b70887-7da6-427b-b3dd-eddf653e493e None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:24.904364 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 151/284] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:24 2022] GET /identity/v3/users/7e4a75e268104e27a0feae94e065971e => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:24.911856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942db900-1c4e-487d-942f-f1ed1b080091 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:25.121520 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942db900-1c4e-487d-942f-f1ed1b080091 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:25.170594 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f42a22-5e83-40d5-8d01-c5313da202bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:25.172181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f42a22-5e83-40d5-8d01-c5313da202bf None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:25.172461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f42a22-5e83-40d5-8d01-c5313da202bf None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:25.172740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f42a22-5e83-40d5-8d01-c5313da202bf None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:25.173979 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f42a22-5e83-40d5-8d01-c5313da202bf None admin] RBAC: Authorizing `identity:get_project(project_id=b05339ac8c27421a9a1921726dcbdc7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:25.175460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f42a22-5e83-40d5-8d01-c5313da202bf None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:25.177162 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 134/285] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:24 2022] GET /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:25.183216 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b70887-7da6-427b-b3dd-eddf653e493e None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:25.226982 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:25.311477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b70887-7da6-427b-b3dd-eddf653e493e None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:25.353929 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35576b2-9063-4167-8d5e-4da9cd41efb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:25.354851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35576b2-9063-4167-8d5e-4da9cd41efb0 None admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:25.355136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35576b2-9063-4167-8d5e-4da9cd41efb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:25.355410 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35576b2-9063-4167-8d5e-4da9cd41efb0 None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f/users/7e4a75e268104e27a0feae94e065971e/roles/c8e4f6d521b04ad39661b74575a8b9d3` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:25.357386 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35576b2-9063-4167-8d5e-4da9cd41efb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b05339ac8c27421a9a1921726dcbdc7f, user_id=7e4a75e268104e27a0feae94e065971e, role_id=c8e4f6d521b04ad39661b74575a8b9d3)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:25.359191 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35576b2-9063-4167-8d5e-4da9cd41efb0 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:25.386682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e856def-175c-4c18-bf6b-95941fd6ce79 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:25.386982 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e856def-175c-4c18-bf6b-95941fd6ce79 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:25.387178 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e856def-175c-4c18-bf6b-95941fd6ce79 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:25.388554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 135/286] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:25:25.394129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b823c-5db5-4db7-80b3-42a9b8b06fb9 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:25.394421 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b823c-5db5-4db7-80b3-42a9b8b06fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:25.394731 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b823c-5db5-4db7-80b3-42a9b8b06fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:25.412925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-af5b823c-5db5-4db7-80b3-42a9b8b06fb9 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:25.414097 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-af5b823c-5db5-4db7-80b3-42a9b8b06fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:25.441486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 152/287] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:25:25 2022] PUT /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f/users/7e4a75e268104e27a0feae94e065971e/roles/c8e4f6d521b04ad39661b74575a8b9d3 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:25:25.480755 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:25.505806 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 136/288] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:25:25.531190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35576b2-9063-4167-8d5e-4da9cd41efb0 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:25.635167 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:25.656924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35576b2-9063-4167-8d5e-4da9cd41efb0 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:25.703069 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faceae1b-9369-413a-99e9-c6b6c8a936e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:25.703880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faceae1b-9369-413a-99e9-c6b6c8a936e2 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:25.704165 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faceae1b-9369-413a-99e9-c6b6c8a936e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:25.704446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faceae1b-9369-413a-99e9-c6b6c8a936e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:25.714898 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faceae1b-9369-413a-99e9-c6b6c8a936e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:25.722804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-faceae1b-9369-413a-99e9-c6b6c8a936e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:25.722804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:25.722804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:25.722804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:25.722804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:25.722804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:25.722804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:25.722804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:25.722804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:25.722804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:25.722804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:25.722804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:25.722804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:25.722804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:25.722804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:25.722804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:25.723712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:25.723712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:25.723712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:25.723712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:25.723712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:25.723712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:25.723712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:25.723712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:25.723712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:25.723712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:25.723712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:25.723712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:25.723712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:25.723712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:25.723712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:25.723712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:25.724483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:25.724483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:25.724483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:25.724483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:25.724483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:25.724483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:25.724483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:25.724483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:25.724483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:25.724483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:25.724483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:25.724483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:25.724483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 153/289] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:25.730576 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b823c-5db5-4db7-80b3-42a9b8b06fb9 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:25.864603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b823c-5db5-4db7-80b3-42a9b8b06fb9 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:25.917277 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e90b8-c28f-45f6-b2eb-8938ee66d473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:25.918039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e90b8-c28f-45f6-b2eb-8938ee66d473 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:25.918201 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e90b8-c28f-45f6-b2eb-8938ee66d473 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:25.918485 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e90b8-c28f-45f6-b2eb-8938ee66d473 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:25.919325 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e90b8-c28f-45f6-b2eb-8938ee66d473 None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:25.919639 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e90b8-c28f-45f6-b2eb-8938ee66d473 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:25.921549 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e90b8-c28f-45f6-b2eb-8938ee66d473 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:25.934479 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 137/290] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:25.940334 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faceae1b-9369-413a-99e9-c6b6c8a936e2 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:26.059248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faceae1b-9369-413a-99e9-c6b6c8a936e2 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:26.102064 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11aa89e-cdd6-498b-bfa3-61b845586439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:26.102969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11aa89e-cdd6-498b-bfa3-61b845586439 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:26.103251 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11aa89e-cdd6-498b-bfa3-61b845586439 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:26.103527 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11aa89e-cdd6-498b-bfa3-61b845586439 None admin] PATH_INFO: `/v3/users/7b06f25a47da4c4b80db567a34eae41c` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:26.104742 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11aa89e-cdd6-498b-bfa3-61b845586439 None admin] RBAC: Authorizing `identity:get_user(user_id=7b06f25a47da4c4b80db567a34eae41c)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:26.106201 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11aa89e-cdd6-498b-bfa3-61b845586439 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:26.107973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 154/291] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:25 2022] GET /identity/v3/users/7b06f25a47da4c4b80db567a34eae41c => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:26.114206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e90b8-c28f-45f6-b2eb-8938ee66d473 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:26.175600 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:26.236295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e90b8-c28f-45f6-b2eb-8938ee66d473 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:26.284108 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ab796-2da5-4d43-88e4-3591f5a22150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:26.284764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ab796-2da5-4d43-88e4-3591f5a22150 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:26.285054 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ab796-2da5-4d43-88e4-3591f5a22150 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:26.285332 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ab796-2da5-4d43-88e4-3591f5a22150 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:26.360662 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ab796-2da5-4d43-88e4-3591f5a22150 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:26.362210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ab796-2da5-4d43-88e4-3591f5a22150 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:26.422945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:26.521992 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 138/292] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:26.527538 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11aa89e-cdd6-498b-bfa3-61b845586439 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:26.549406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:26.666497 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11aa89e-cdd6-498b-bfa3-61b845586439 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:26.708096 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74036f-6da8-4335-91cf-d9d47006831a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:26.708970 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74036f-6da8-4335-91cf-d9d47006831a None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:26.709256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74036f-6da8-4335-91cf-d9d47006831a None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:26.709534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74036f-6da8-4335-91cf-d9d47006831a None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:26.715296 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74036f-6da8-4335-91cf-d9d47006831a None admin] RBAC: Authorizing `identity:get_project(project_id=b05339ac8c27421a9a1921726dcbdc7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:26.716782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74036f-6da8-4335-91cf-d9d47006831a None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:26.719431 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 155/293] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:26 2022] GET /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:26.727103 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ab796-2da5-4d43-88e4-3591f5a22150 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:26.846594 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ab796-2da5-4d43-88e4-3591f5a22150 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:26.860715 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa85426-3909-4ad7-916d-a87f08112e5f None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:26.861015 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa85426-3909-4ad7-916d-a87f08112e5f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:26.861276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa85426-3909-4ad7-916d-a87f08112e5f None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:26.862669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 156/294] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:26.867929 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8961ef-7b79-4d6e-8443-9abfcf8ee5d8 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:26.868223 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8961ef-7b79-4d6e-8443-9abfcf8ee5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:26.868504 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8961ef-7b79-4d6e-8443-9abfcf8ee5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:26.894402 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-9f8961ef-7b79-4d6e-8443-9abfcf8ee5d8 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:26.895658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-9f8961ef-7b79-4d6e-8443-9abfcf8ee5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:26.917016 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1129678-7022-434a-acc4-accd877ffb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:26.917707 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1129678-7022-434a-acc4-accd877ffb92 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:26.918224 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1129678-7022-434a-acc4-accd877ffb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:26.918318 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1129678-7022-434a-acc4-accd877ffb92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:26.919216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1129678-7022-434a-acc4-accd877ffb92 None admin] RBAC: Adding query filter params (role.id=75f08cd0671f436c9f968976aa8baa7f, scope.project.id=b05339ac8c27421a9a1921726dcbdc7f, user.id=7b06f25a47da4c4b80db567a34eae41c) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:26.919539 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1129678-7022-434a-acc4-accd877ffb92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:26.921065 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1129678-7022-434a-acc4-accd877ffb92 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:26.928017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 139/295] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:25:26 2022] GET /identity/v3/role_assignments?user.id=7b06f25a47da4c4b80db567a34eae41c&scope.project.id=b05339ac8c27421a9a1921726dcbdc7f&role.id=75f08cd0671f436c9f968976aa8baa7f => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:27.040333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 157/296] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:27.076429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1129678-7022-434a-acc4-accd877ffb92 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:27.097221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:27.124308 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7b198-818f-4102-a510-9b2421d51426 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:27.124679 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7b198-818f-4102-a510-9b2421d51426 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:27.125036 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7b198-818f-4102-a510-9b2421d51426 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:27.126805 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 158/297] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:27.133570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4785579-00e3-4c50-98ab-941a0fe9578e None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:27.135997 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4785579-00e3-4c50-98ab-941a0fe9578e None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:27.135997 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4785579-00e3-4c50-98ab-941a0fe9578e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:27.158955 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-d4785579-00e3-4c50-98ab-941a0fe9578e None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:27.160086 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d4785579-00e3-4c50-98ab-941a0fe9578e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:27.170294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:27.228191 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1129678-7022-434a-acc4-accd877ffb92 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:27.272411 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f028a0-fd08-4da5-8ecf-fc62704c9477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:27.273747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f028a0-fd08-4da5-8ecf-fc62704c9477 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:27.274290 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f028a0-fd08-4da5-8ecf-fc62704c9477 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:27.274395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 159/298] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:25:27.274902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f028a0-fd08-4da5-8ecf-fc62704c9477 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:27.287031 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f028a0-fd08-4da5-8ecf-fc62704c9477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:27.295440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-30f028a0-fd08-4da5-8ecf-fc62704c9477 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:27.295440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:27.295440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:27.295440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:27.295440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:27.295440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:27.295440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:27.295440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:27.295440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:27.295440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:27.295440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:27.295440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:27.295440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:27.295440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:27.295440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:27.295440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:27.296880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:27.296880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:27.296880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:27.296880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:27.296880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:27.296880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:27.296880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:27.296880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:27.296880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:27.296880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:27.296880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:27.296880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:27.296880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:27.296880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:27.296880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:27.296880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:27.297694 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:27.297694 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:27.297694 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:27.297694 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:27.297694 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:27.297694 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:27.297694 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:27.297694 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:27.297694 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:27.297694 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:27.297694 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:27.297694 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:27.300465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 140/299] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:27.302223 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4785579-00e3-4c50-98ab-941a0fe9578e None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:27.306224 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f028a0-fd08-4da5-8ecf-fc62704c9477 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:27.387084 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:27.418405 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f028a0-fd08-4da5-8ecf-fc62704c9477 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:27.425549 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4785579-00e3-4c50-98ab-941a0fe9578e None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:27.455008 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28947e-c466-440c-ab9b-4ca7d5ce3948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:27.456007 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28947e-c466-440c-ab9b-4ca7d5ce3948 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:27.456439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28947e-c466-440c-ab9b-4ca7d5ce3948 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:27.456891 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28947e-c466-440c-ab9b-4ca7d5ce3948 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:27.457904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28947e-c466-440c-ab9b-4ca7d5ce3948 None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:27.458452 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28947e-c466-440c-ab9b-4ca7d5ce3948 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:27.460148 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28947e-c466-440c-ab9b-4ca7d5ce3948 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:27.462822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d86db-60f3-411e-89a6-996b33a3e0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:27.463615 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d86db-60f3-411e-89a6-996b33a3e0fa None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:27.463897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d86db-60f3-411e-89a6-996b33a3e0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:27.464170 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d86db-60f3-411e-89a6-996b33a3e0fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:27.465364 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d86db-60f3-411e-89a6-996b33a3e0fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:27.466821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d86db-60f3-411e-89a6-996b33a3e0fa None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:27.468578 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 160/300] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:27.469864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475afb3-4fdc-4597-bba2-ae63a0c3b27a None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:27.470149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475afb3-4fdc-4597-bba2-ae63a0c3b27a None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:27.470425 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475afb3-4fdc-4597-bba2-ae63a0c3b27a None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:27.470800 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 141/301] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:27.474184 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28947e-c466-440c-ab9b-4ca7d5ce3948 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:27.476047 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 161/302] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:27.476795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475afb3-4fdc-4597-bba2-ae63a0c3b27a None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:27.495258 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:27.590608 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28947e-c466-440c-ab9b-4ca7d5ce3948 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:27.597512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475afb3-4fdc-4597-bba2-ae63a0c3b27a None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:27.622453 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:27.629239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293caf52-e525-43e7-a8bf-bc2eb94c6694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:27.629989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293caf52-e525-43e7-a8bf-bc2eb94c6694 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:27.630273 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293caf52-e525-43e7-a8bf-bc2eb94c6694 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:27.630576 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293caf52-e525-43e7-a8bf-bc2eb94c6694 None admin] PATH_INFO: `/v3/groups` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:27.631265 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293caf52-e525-43e7-a8bf-bc2eb94c6694 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:27.632715 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293caf52-e525-43e7-a8bf-bc2eb94c6694 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:27.633980 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcad8be-a2a6-4205-8c10-646ec43f3e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:27.634874 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcad8be-a2a6-4205-8c10-646ec43f3e9b None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:27.635155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcad8be-a2a6-4205-8c10-646ec43f3e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:27.635656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcad8be-a2a6-4205-8c10-646ec43f3e9b None admin] PATH_INFO: `/v3/users/2cda5981dcd54219b7d697b973badf36` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:27.636889 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcad8be-a2a6-4205-8c10-646ec43f3e9b None admin] RBAC: Authorizing `identity:get_user(user_id=2cda5981dcd54219b7d697b973badf36)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:27.638856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcad8be-a2a6-4205-8c10-646ec43f3e9b None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:27.640815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 162/303] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:27 2022] GET /identity/v3/users/2cda5981dcd54219b7d697b973badf36 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:27.643466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c5b08-c388-41f1-9a4a-026d6ed4c842 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:27.643466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c5b08-c388-41f1-9a4a-026d6ed4c842 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:27.643466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c5b08-c388-41f1-9a4a-026d6ed4c842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:27.659359 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-0b8c5b08-c388-41f1-9a4a-026d6ed4c842 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:27.660576 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-0b8c5b08-c388-41f1-9a4a-026d6ed4c842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:27.714481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 142/304] 10.222.0.13 () {60 vars in 1134 bytes} [Thu Jun 23 11:25:27 2022] POST /identity/v3/groups => generated 229 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:25:27.727603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293caf52-e525-43e7-a8bf-bc2eb94c6694 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:27.728225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 163/305] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:27.753387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c5b08-c388-41f1-9a4a-026d6ed4c842 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:27.764326 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:27.870784 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c5b08-c388-41f1-9a4a-026d6ed4c842 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:27.873558 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293caf52-e525-43e7-a8bf-bc2eb94c6694 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:27.911396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467bdd1f-120f-4157-b78f-4c61a28facff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:27.912202 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bdd1f-120f-4157-b78f-4c61a28facff None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:27.912444 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bdd1f-120f-4157-b78f-4c61a28facff None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:27.912716 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bdd1f-120f-4157-b78f-4c61a28facff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:27.917571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4068aa-ec2f-4d98-8096-cbb48e4876be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:27.918442 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4068aa-ec2f-4d98-8096-cbb48e4876be None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:27.919006 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4068aa-ec2f-4d98-8096-cbb48e4876be None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:27.919341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4068aa-ec2f-4d98-8096-cbb48e4876be None admin] PATH_INFO: `/v3/system/users/2cda5981dcd54219b7d697b973badf36/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:27.923526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4068aa-ec2f-4d98-8096-cbb48e4876be None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2cda5981dcd54219b7d697b973badf36, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:27.923724 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4068aa-ec2f-4d98-8096-cbb48e4876be None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:27.925271 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467bdd1f-120f-4157-b78f-4c61a28facff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:27.933944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-467bdd1f-120f-4157-b78f-4c61a28facff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:25:27.933944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:27.933944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:27.933944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:27.933944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:27.933944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:27.933944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:27.933944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:27.933944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:27.933944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:27.933944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:27.933944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:27.933944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:27.933944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:27.933944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:27.933944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:27.940608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:27.940608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:27.940608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:27.940608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:27.940608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:27.940608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:27.940608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:27.940608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:27.940608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:27.940608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:27.940608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:27.940608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:27.940608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:27.940608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:27.940608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:27.940608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:27.941396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:27.941396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:27.941396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:27.941396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:27.941396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:27.941396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:27.941396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:27.941396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:27.941396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:27.941396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:27.941396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:25:27.941396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:27.941396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 164/306] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:27.943554 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467bdd1f-120f-4157-b78f-4c61a28facff None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:28.014109 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:28.029439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 143/307] 10.222.0.13 () {58 vars in 1334 bytes} [Thu Jun 23 11:25:27 2022] PUT /identity/v3/system/users/2cda5981dcd54219b7d697b973badf36/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:25:28.033463 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d72843-d522-4e6a-a011-8350396f1ea9 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:28.033463 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d72843-d522-4e6a-a011-8350396f1ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:28.033463 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d72843-d522-4e6a-a011-8350396f1ea9 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:28.033463 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 144/308] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:28.039445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e625a89-4e46-4b84-9935-0b40ad2eb7af None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:28.039737 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e625a89-4e46-4b84-9935-0b40ad2eb7af None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:28.040082 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e625a89-4e46-4b84-9935-0b40ad2eb7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:28.057420 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467bdd1f-120f-4157-b78f-4c61a28facff None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:28.059912 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-9e625a89-4e46-4b84-9935-0b40ad2eb7af None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:28.063482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9e625a89-4e46-4b84-9935-0b40ad2eb7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:28.106002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1b421-bd3d-442d-9e48-e79031e63ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:28.106999 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1b421-bd3d-442d-9e48-e79031e63ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:28.107421 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1b421-bd3d-442d-9e48-e79031e63ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:28.107834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1b421-bd3d-442d-9e48-e79031e63ff1 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:28.109611 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1b421-bd3d-442d-9e48-e79031e63ff1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:28.110086 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1b421-bd3d-442d-9e48-e79031e63ff1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:28.111702 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1b421-bd3d-442d-9e48-e79031e63ff1 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:28.121968 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 165/309] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:28.127603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1b421-bd3d-442d-9e48-e79031e63ff1 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:28.147660 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:28.167067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 145/310] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:25:28.192313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e625a89-4e46-4b84-9935-0b40ad2eb7af None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:28.262677 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1b421-bd3d-442d-9e48-e79031e63ff1 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:28.294096 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:28.300658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9078e-d2a3-4b6a-ad71-b968ad3232d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:28.301770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9078e-d2a3-4b6a-ad71-b968ad3232d9 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:28.302334 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9078e-d2a3-4b6a-ad71-b968ad3232d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:28.302870 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9078e-d2a3-4b6a-ad71-b968ad3232d9 None admin] PATH_INFO: `/v3/users/1913d1b97f0f468f9686c3ac9537b446` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:28.317294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9078e-d2a3-4b6a-ad71-b968ad3232d9 None admin] RBAC: Authorizing `identity:get_user(user_id=1913d1b97f0f468f9686c3ac9537b446)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:28.319438 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9078e-d2a3-4b6a-ad71-b968ad3232d9 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:28.321704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 166/311] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:28 2022] GET /identity/v3/users/1913d1b97f0f468f9686c3ac9537b446 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:28.328366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9078e-d2a3-4b6a-ad71-b968ad3232d9 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:28.337996 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e625a89-4e46-4b84-9935-0b40ad2eb7af None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:28.378739 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be09665-4165-49c0-adef-5a717bbbafed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:28.379537 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be09665-4165-49c0-adef-5a717bbbafed None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:28.381108 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be09665-4165-49c0-adef-5a717bbbafed None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:28.381108 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be09665-4165-49c0-adef-5a717bbbafed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:28.395764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be09665-4165-49c0-adef-5a717bbbafed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:28.403582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-1be09665-4165-49c0-adef-5a717bbbafed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:25:28.403582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:28.403582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:28.403582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:28.403582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:28.403582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:28.403582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:28.403582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:28.403582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:28.403582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:28.403582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:28.403582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:28.403582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:28.403582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:28.403582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:28.403582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:28.404749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:28.404749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:28.404749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:28.404749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:28.404749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:28.404749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:28.404749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:28.404749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:28.404749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:28.404749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:28.404749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:28.404749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:28.404749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:28.404749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:28.404749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:28.404749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:28.406924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:28.406924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:28.406924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:28.406924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:28.406924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:28.406924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:28.406924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:28.406924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:28.406924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:28.406924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:28.406924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:25:28.406924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:28.410272 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 146/312] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:28.411328 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be09665-4165-49c0-adef-5a717bbbafed None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:28.461313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:28.477506 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9078e-d2a3-4b6a-ad71-b968ad3232d9 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:28.523762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce7dbb-173a-43c5-9a75-62479c6e7a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:28.524812 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce7dbb-173a-43c5-9a75-62479c6e7a9b None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:28.525290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce7dbb-173a-43c5-9a75-62479c6e7a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:28.525780 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce7dbb-173a-43c5-9a75-62479c6e7a9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:28.526838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce7dbb-173a-43c5-9a75-62479c6e7a9b None admin] RBAC: Adding query filter params (role.id=c8e4f6d521b04ad39661b74575a8b9d3, user.id=1913d1b97f0f468f9686c3ac9537b446, scope.system=all) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:28.527350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce7dbb-173a-43c5-9a75-62479c6e7a9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:28.528992 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce7dbb-173a-43c5-9a75-62479c6e7a9b None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:28.534002 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be09665-4165-49c0-adef-5a717bbbafed None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:28.542648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 167/313] 10.222.0.13 () {56 vars in 1308 bytes} [Thu Jun 23 11:25:28 2022] GET /identity/v3/role_assignments?user.id=1913d1b97f0f468f9686c3ac9537b446&scope.system=all&role.id=c8e4f6d521b04ad39661b74575a8b9d3 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:28.574938 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c49b5f-ea5a-4459-bd82-e92dfc381745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:28.575619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c49b5f-ea5a-4459-bd82-e92dfc381745 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:28.575958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c49b5f-ea5a-4459-bd82-e92dfc381745 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:28.576236 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c49b5f-ea5a-4459-bd82-e92dfc381745 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:28.577001 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c49b5f-ea5a-4459-bd82-e92dfc381745 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:28.578112 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c49b5f-ea5a-4459-bd82-e92dfc381745 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:28.579570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c49b5f-ea5a-4459-bd82-e92dfc381745 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:28.589795 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 147/314] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:28.595520 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce7dbb-173a-43c5-9a75-62479c6e7a9b None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:28.662911 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:28.709847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce7dbb-173a-43c5-9a75-62479c6e7a9b None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:28.746168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec01bdf-86ee-4e41-8527-5cb274bc96fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:28.746986 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec01bdf-86ee-4e41-8527-5cb274bc96fa None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:28.747260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec01bdf-86ee-4e41-8527-5cb274bc96fa None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:28.747528 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec01bdf-86ee-4e41-8527-5cb274bc96fa None admin] PATH_INFO: `/v3/users/7e4a75e268104e27a0feae94e065971e` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:28.748782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec01bdf-86ee-4e41-8527-5cb274bc96fa None admin] RBAC: Authorizing `identity:get_user(user_id=7e4a75e268104e27a0feae94e065971e)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:28.750952 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec01bdf-86ee-4e41-8527-5cb274bc96fa None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:28.752941 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 168/315] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:28 2022] GET /identity/v3/users/7e4a75e268104e27a0feae94e065971e => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:28.758494 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c49b5f-ea5a-4459-bd82-e92dfc381745 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:28.867199 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c49b5f-ea5a-4459-bd82-e92dfc381745 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:28.877431 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:28.903892 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6654f-9693-4632-b07a-959d6b45384c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:28.904545 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6654f-9693-4632-b07a-959d6b45384c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:28.904866 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6654f-9693-4632-b07a-959d6b45384c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:28.905265 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6654f-9693-4632-b07a-959d6b45384c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:28.978032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e6654f-9693-4632-b07a-959d6b45384c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:28.979614 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e6654f-9693-4632-b07a-959d6b45384c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:29.082420 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 148/316] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:29.087849 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec01bdf-86ee-4e41-8527-5cb274bc96fa None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:29.190042 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec01bdf-86ee-4e41-8527-5cb274bc96fa None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:29.215325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:29.232174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0fa55-afc9-4e56-b72a-a2d9a5f46593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:29.234015 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0fa55-afc9-4e56-b72a-a2d9a5f46593 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:29.242821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0fa55-afc9-4e56-b72a-a2d9a5f46593 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:29.242821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0fa55-afc9-4e56-b72a-a2d9a5f46593 None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:29.242821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0fa55-afc9-4e56-b72a-a2d9a5f46593 None admin] RBAC: Authorizing `identity:get_project(project_id=b05339ac8c27421a9a1921726dcbdc7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:29.242821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0fa55-afc9-4e56-b72a-a2d9a5f46593 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:29.242821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 169/317] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:29 2022] GET /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:29.245848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6654f-9693-4632-b07a-959d6b45384c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:29.259290 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:29.353665 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6654f-9693-4632-b07a-959d6b45384c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:29.392067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9cff5d-6ddd-4861-9062-30a446303a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:29.392726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9cff5d-6ddd-4861-9062-30a446303a4c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:29.393008 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9cff5d-6ddd-4861-9062-30a446303a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:29.393279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9cff5d-6ddd-4861-9062-30a446303a4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:29.394167 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9cff5d-6ddd-4861-9062-30a446303a4c None admin] RBAC: Adding query filter params (role.id=c8e4f6d521b04ad39661b74575a8b9d3, scope.project.id=b05339ac8c27421a9a1921726dcbdc7f, user.id=7e4a75e268104e27a0feae94e065971e) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:29.394479 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9cff5d-6ddd-4861-9062-30a446303a4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:29.396053 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9cff5d-6ddd-4861-9062-30a446303a4c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:29.402428 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 149/318] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:25:29 2022] GET /identity/v3/role_assignments?user.id=7e4a75e268104e27a0feae94e065971e&scope.project.id=b05339ac8c27421a9a1921726dcbdc7f&role.id=c8e4f6d521b04ad39661b74575a8b9d3 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:29.856522 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd9111-37de-4eda-80ab-4c4d647f70bf None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:29.856899 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd9111-37de-4eda-80ab-4c4d647f70bf None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:29.857080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd9111-37de-4eda-80ab-4c4d647f70bf None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:29.859210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 170/319] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:29.864350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1e46a-ac1e-4392-a16b-8c2d18ff28a2 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:29.864648 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1e46a-ac1e-4392-a16b-8c2d18ff28a2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:29.864926 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1e46a-ac1e-4392-a16b-8c2d18ff28a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:29.882022 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-cea1e46a-ac1e-4392-a16b-8c2d18ff28a2 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:29.883211 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-cea1e46a-ac1e-4392-a16b-8c2d18ff28a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:29.948613 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 150/320] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:29.971100 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd9111-37de-4eda-80ab-4c4d647f70bf None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:30.071755 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd9111-37de-4eda-80ab-4c4d647f70bf None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:30.105633 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc87e3e-4d8e-44a7-bf63-bfb988f05173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:30.106428 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc87e3e-4d8e-44a7-bf63-bfb988f05173 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:30.106795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc87e3e-4d8e-44a7-bf63-bfb988f05173 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:30.107018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc87e3e-4d8e-44a7-bf63-bfb988f05173 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:30.108174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc87e3e-4d8e-44a7-bf63-bfb988f05173 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:30.110164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc87e3e-4d8e-44a7-bf63-bfb988f05173 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:30.111989 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 171/321] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:30.117378 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea1e46a-ac1e-4392-a16b-8c2d18ff28a2 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:30.120507 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:30.175103 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b5257-d22f-491d-a16f-e0deed0cddef None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:30.175392 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b5257-d22f-491d-a16f-e0deed0cddef None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:30.175665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b5257-d22f-491d-a16f-e0deed0cddef None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:30.177194 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 172/322] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:30.181752 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f295d-62cd-404b-9367-2dab52d7fac5 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:30.182040 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f295d-62cd-404b-9367-2dab52d7fac5 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:30.182321 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f295d-62cd-404b-9367-2dab52d7fac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:30.199337 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c39f295d-62cd-404b-9367-2dab52d7fac5 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:30.200505 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c39f295d-62cd-404b-9367-2dab52d7fac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:30.220769 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea1e46a-ac1e-4392-a16b-8c2d18ff28a2 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:30.253827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:30.256557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8282a7-dab7-480e-bb12-8bc4f41bc73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:30.257338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8282a7-dab7-480e-bb12-8bc4f41bc73b None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:30.257623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8282a7-dab7-480e-bb12-8bc4f41bc73b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:30.257895 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8282a7-dab7-480e-bb12-8bc4f41bc73b None admin] PATH_INFO: `/v3/groups` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:30.259033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8282a7-dab7-480e-bb12-8bc4f41bc73b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:30.260121 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8282a7-dab7-480e-bb12-8bc4f41bc73b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:30.271150 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 173/323] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:25:30.294008 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f295d-62cd-404b-9367-2dab52d7fac5 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:30.396941 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f295d-62cd-404b-9367-2dab52d7fac5 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:30.398919 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 151/324] 10.222.0.13 () {60 vars in 1134 bytes} [Thu Jun 23 11:25:30 2022] POST /identity/v3/groups => generated 226 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:25:30.435555 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f77f4-d4bd-4b20-b3d0-a7dfdaaa4945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:30.436417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f77f4-d4bd-4b20-b3d0-a7dfdaaa4945 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:30.437308 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f77f4-d4bd-4b20-b3d0-a7dfdaaa4945 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:30.437535 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f77f4-d4bd-4b20-b3d0-a7dfdaaa4945 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:30.450117 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f77f4-d4bd-4b20-b3d0-a7dfdaaa4945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:30.458229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-2a3f77f4-d4bd-4b20-b3d0-a7dfdaaa4945 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:30.458229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:30.458229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:30.458229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:30.458229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:30.458229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:30.458229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:30.458229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:30.458229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:30.458229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:30.458229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:30.458229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:30.458229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:30.458229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:30.458229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:30.458229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:30.459237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:30.459237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:30.459237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:30.459237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:30.459237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:30.459237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:30.459237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:30.459237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:30.459237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:30.459237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:30.459237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:30.459237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:30.459237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:30.459237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:30.459237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:30.459237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:30.460063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:30.460063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:30.460063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:30.460063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:30.460063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:30.460063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:30.460063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:30.460063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:30.460063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:30.460063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:30.460063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:30.460063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:30.462880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 174/325] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:25:30.465355 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8282a7-dab7-480e-bb12-8bc4f41bc73b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:30.541035 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:30.584161 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8282a7-dab7-480e-bb12-8bc4f41bc73b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:30.619625 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb680e-7f6b-4796-9397-790846bea400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:30.620337 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb680e-7f6b-4796-9397-790846bea400 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:30.620583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb680e-7f6b-4796-9397-790846bea400 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:30.620862 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb680e-7f6b-4796-9397-790846bea400 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:30.621616 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb680e-7f6b-4796-9397-790846bea400 None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:30.621929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb680e-7f6b-4796-9397-790846bea400 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:30.623358 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb680e-7f6b-4796-9397-790846bea400 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:30.634438 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 152/326] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:30.640486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f77f4-d4bd-4b20-b3d0-a7dfdaaa4945 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:30.678922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afaaa2-1f33-4b83-9bd3-c10cec7d000e None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:30.679214 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afaaa2-1f33-4b83-9bd3-c10cec7d000e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:30.679401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afaaa2-1f33-4b83-9bd3-c10cec7d000e None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:30.680822 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 153/327] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:25:30.687390 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718279fa-029d-45b9-ac29-f09def79e068 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:30.687703 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718279fa-029d-45b9-ac29-f09def79e068 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:30.687987 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718279fa-029d-45b9-ac29-f09def79e068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:30.694928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:30.710092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-718279fa-029d-45b9-ac29-f09def79e068 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:30.711308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-718279fa-029d-45b9-ac29-f09def79e068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:30.771876 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f77f4-d4bd-4b20-b3d0-a7dfdaaa4945 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:30.811985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 154/328] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:30.817931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a91ccf-771c-48a7-a69d-91a36c53e61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:30.819324 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a91ccf-771c-48a7-a69d-91a36c53e61a None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:30.819803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a91ccf-771c-48a7-a69d-91a36c53e61a None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:30.820294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a91ccf-771c-48a7-a69d-91a36c53e61a None admin] PATH_INFO: `/v3/users/2cda5981dcd54219b7d697b973badf36` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:30.821731 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a91ccf-771c-48a7-a69d-91a36c53e61a None admin] RBAC: Authorizing `identity:get_user(user_id=2cda5981dcd54219b7d697b973badf36)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:30.823426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a91ccf-771c-48a7-a69d-91a36c53e61a None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:30.825539 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 175/329] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:30 2022] GET /identity/v3/users/2cda5981dcd54219b7d697b973badf36 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:30.831817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718279fa-029d-45b9-ac29-f09def79e068 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:30.839942 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a91ccf-771c-48a7-a69d-91a36c53e61a None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:30.909758 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:30.965810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a91ccf-771c-48a7-a69d-91a36c53e61a None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:30.968158 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718279fa-029d-45b9-ac29-f09def79e068 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:30.981902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:31.005455 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5abd0-3688-45d8-98d6-dd592566d8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:31.006457 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5abd0-3688-45d8-98d6-dd592566d8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:31.006778 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5abd0-3688-45d8-98d6-dd592566d8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:31.007060 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5abd0-3688-45d8-98d6-dd592566d8a9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:31.007171 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292469dc-2196-4d0f-adbb-bcc504e09e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:31.007757 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292469dc-2196-4d0f-adbb-bcc504e09e25 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:31.008039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292469dc-2196-4d0f-adbb-bcc504e09e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:31.008313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292469dc-2196-4d0f-adbb-bcc504e09e25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:31.009152 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292469dc-2196-4d0f-adbb-bcc504e09e25 None admin] RBAC: Adding query filter params (role.id=75f08cd0671f436c9f968976aa8baa7f, user.id=2cda5981dcd54219b7d697b973badf36, scope.system=all) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:31.009463 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292469dc-2196-4d0f-adbb-bcc504e09e25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:31.010933 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292469dc-2196-4d0f-adbb-bcc504e09e25 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:31.018770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca5abd0-3688-45d8-98d6-dd592566d8a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:31.023707 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 155/330] 10.222.0.13 () {56 vars in 1308 bytes} [Thu Jun 23 11:25:30 2022] GET /identity/v3/role_assignments?user.id=2cda5981dcd54219b7d697b973badf36&scope.system=all&role.id=75f08cd0671f436c9f968976aa8baa7f => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:31.034728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-7ca5abd0-3688-45d8-98d6-dd592566d8a9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:25:31.034728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:31.034728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:31.034728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:31.034728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:31.034728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:31.034728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:31.034728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:31.034728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:31.034728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:31.034728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:31.034728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:31.034728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:31.034728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:31.034728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:31.034728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:31.035681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:31.035681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:31.035681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:31.035681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:31.035681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:31.035681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:31.035681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:31.035681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:31.035681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:31.035681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:31.035681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:31.035681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:31.035681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:31.035681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:31.035681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:31.035681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:31.036448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:31.036448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:31.036448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:31.036448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:31.036448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:31.036448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:31.036448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:31.036448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:31.036448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:31.036448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:31.036448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:25:31.036448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:31.037049 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 176/331] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:25:31.043095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292469dc-2196-4d0f-adbb-bcc504e09e25 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:31.183279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292469dc-2196-4d0f-adbb-bcc504e09e25 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:31.198978 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:31.227243 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b949693-ce88-408e-a621-b2137e611062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:31.227913 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b949693-ce88-408e-a621-b2137e611062 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:31.228206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b949693-ce88-408e-a621-b2137e611062 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:31.228487 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b949693-ce88-408e-a621-b2137e611062 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:31.229249 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b949693-ce88-408e-a621-b2137e611062 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:31.229562 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b949693-ce88-408e-a621-b2137e611062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:31.231768 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b949693-ce88-408e-a621-b2137e611062 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:31.245937 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 156/332] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:31.251742 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5abd0-3688-45d8-98d6-dd592566d8a9 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:31.378775 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5abd0-3688-45d8-98d6-dd592566d8a9 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:31.415513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08a26d-cbc4-4df4-ba28-2b002c48bbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:31.416766 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08a26d-cbc4-4df4-ba28-2b002c48bbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:31.417262 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08a26d-cbc4-4df4-ba28-2b002c48bbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:31.417727 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08a26d-cbc4-4df4-ba28-2b002c48bbe7 None admin] PATH_INFO: `/v3/users/1913d1b97f0f468f9686c3ac9537b446` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:31.419141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08a26d-cbc4-4df4-ba28-2b002c48bbe7 None admin] RBAC: Authorizing `identity:get_user(user_id=1913d1b97f0f468f9686c3ac9537b446)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:31.420851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08a26d-cbc4-4df4-ba28-2b002c48bbe7 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:31.422998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 177/333] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:31 2022] GET /identity/v3/users/1913d1b97f0f468f9686c3ac9537b446 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:31.429821 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b949693-ce88-408e-a621-b2137e611062 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:31.550456 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b949693-ce88-408e-a621-b2137e611062 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:31.591004 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f0282-068c-44e4-8aa5-c607a03dc2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:31.592264 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f0282-068c-44e4-8aa5-c607a03dc2c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:31.592816 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f0282-068c-44e4-8aa5-c607a03dc2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:31.593288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f0282-068c-44e4-8aa5-c607a03dc2c4 None admin] PATH_INFO: `/v3/system/users/1913d1b97f0f468f9686c3ac9537b446/roles/c8e4f6d521b04ad39661b74575a8b9d3` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:31.595770 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f0282-068c-44e4-8aa5-c607a03dc2c4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1913d1b97f0f468f9686c3ac9537b446, role_id=c8e4f6d521b04ad39661b74575a8b9d3)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:31.597353 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f0282-068c-44e4-8aa5-c607a03dc2c4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:31.669659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 157/334] 10.222.0.13 () {58 vars in 1334 bytes} [Thu Jun 23 11:25:31 2022] PUT /identity/v3/system/users/1913d1b97f0f468f9686c3ac9537b446/roles/c8e4f6d521b04ad39661b74575a8b9d3 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:25:33.060956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8d892-efb2-43e2-9280-f2ae56dbacb3 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:33.062105 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8d892-efb2-43e2-9280-f2ae56dbacb3 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:33.062232 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8d892-efb2-43e2-9280-f2ae56dbacb3 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:33.063826 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 178/335] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:33.069433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab2597-aca0-4ac3-845c-767956504cec None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:33.069856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab2597-aca0-4ac3-845c-767956504cec None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:33.070048 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab2597-aca0-4ac3-845c-767956504cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:33.091804 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-38ab2597-aca0-4ac3-845c-767956504cec None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:33.093820 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-38ab2597-aca0-4ac3-845c-767956504cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:33.129014 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:33.172288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 158/336] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:33.211274 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8d892-efb2-43e2-9280-f2ae56dbacb3 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:33.277975 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:33.304225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681fccc-e6c4-4f5f-a7ca-3028a4ac77b5 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:33.304559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681fccc-e6c4-4f5f-a7ca-3028a4ac77b5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:33.304788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681fccc-e6c4-4f5f-a7ca-3028a4ac77b5 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:33.306409 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 159/337] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:33.312469 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abbc95-48d6-4fbb-94af-e97c1d2e94de None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:33.312596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abbc95-48d6-4fbb-94af-e97c1d2e94de None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:33.312817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abbc95-48d6-4fbb-94af-e97c1d2e94de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:33.338709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b5abbc95-48d6-4fbb-94af-e97c1d2e94de None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:33.339912 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b5abbc95-48d6-4fbb-94af-e97c1d2e94de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:33.362173 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8d892-efb2-43e2-9280-f2ae56dbacb3 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:33.442098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d58d6d-a07a-480e-bc21-abd18889cda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:33.442963 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d58d6d-a07a-480e-bc21-abd18889cda4 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:33.443506 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d58d6d-a07a-480e-bc21-abd18889cda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:33.443786 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d58d6d-a07a-480e-bc21-abd18889cda4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:33.471620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d58d6d-a07a-480e-bc21-abd18889cda4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:33.488161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-60d58d6d-a07a-480e-bc21-abd18889cda4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:33.488161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:33.488161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:33.488161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:33.488161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:33.488161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:33.488161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:33.488161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:33.488161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:33.488161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:33.488161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:33.488161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:33.488161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:33.488161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:33.488161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:33.488161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:33.489035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:33.489035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:33.489035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:33.489035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:33.489035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:33.489035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:33.489035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:33.489035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:33.489035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:33.489035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:33.489035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:33.489035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:33.489035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:33.489035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:33.489035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:33.489035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:33.489831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:33.489831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:33.489831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:33.489831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:33.489831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:33.489831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:33.489831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:33.489831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:33.489831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:33.489831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:33.489831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:33.489831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:33.489831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 179/338] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:25:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:33.498592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d58d6d-a07a-480e-bc21-abd18889cda4 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:33.515426 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 160/339] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:25:33.543270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5abbc95-48d6-4fbb-94af-e97c1d2e94de None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:33.622730 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d58d6d-a07a-480e-bc21-abd18889cda4 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:33.624343 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:33.710341 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb284ff0-1ec1-43e1-949b-1b57e15096be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:33.711054 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb284ff0-1ec1-43e1-949b-1b57e15096be None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:33.711341 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb284ff0-1ec1-43e1-949b-1b57e15096be None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:33.711615 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb284ff0-1ec1-43e1-949b-1b57e15096be None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:33.712369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb284ff0-1ec1-43e1-949b-1b57e15096be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:33.712678 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb284ff0-1ec1-43e1-949b-1b57e15096be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:33.714061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb284ff0-1ec1-43e1-949b-1b57e15096be None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:33.729388 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 180/340] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:25:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:33.737107 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb284ff0-1ec1-43e1-949b-1b57e15096be None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:33.762958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:33.769619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5abbc95-48d6-4fbb-94af-e97c1d2e94de None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:33.870506 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa9d6f-ae55-453d-b1fe-c78e78929dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:33.871366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa9d6f-ae55-453d-b1fe-c78e78929dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:33.871652 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa9d6f-ae55-453d-b1fe-c78e78929dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:33.871925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa9d6f-ae55-453d-b1fe-c78e78929dd3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:33.911655 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa9d6f-ae55-453d-b1fe-c78e78929dd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:33.919518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-88fa9d6f-ae55-453d-b1fe-c78e78929dd3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:33.919518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:33.919518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:33.919518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:33.919518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:33.919518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:33.919518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:33.919518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:33.919518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:33.919518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:33.919518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:33.919518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:33.919518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:33.919518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:33.919518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:33.919518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:33.920394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:33.920394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:33.920394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:33.920394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:33.920394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:33.920394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:33.920394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:33.920394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:33.920394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:33.920394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:33.920394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:33.920394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:33.920394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:33.920394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:33.920394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:33.920394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:33.921216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:33.921216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:33.921216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:33.921216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:33.921216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:33.921216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:33.921216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:33.921216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:33.921216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:33.921216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:33.921216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:33.921216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:33.921819 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 161/341] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:33.927194 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa9d6f-ae55-453d-b1fe-c78e78929dd3 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:33.938491 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb284ff0-1ec1-43e1-949b-1b57e15096be None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:33.988513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bee5dd-1934-4a88-ad7b-14f30713da69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:33.989398 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bee5dd-1934-4a88-ad7b-14f30713da69 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:33.989687 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bee5dd-1934-4a88-ad7b-14f30713da69 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:33.989970 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bee5dd-1934-4a88-ad7b-14f30713da69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:34.046915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:34.100137 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bee5dd-1934-4a88-ad7b-14f30713da69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:34.101802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bee5dd-1934-4a88-ad7b-14f30713da69 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:34.173748 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa9d6f-ae55-453d-b1fe-c78e78929dd3 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:34.213937 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 181/342] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:34.215414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87652a6-1a61-4a23-9244-261d66a9b3ae None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:34.216117 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87652a6-1a61-4a23-9244-261d66a9b3ae None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:34.216405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87652a6-1a61-4a23-9244-261d66a9b3ae None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:34.218938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 182/343] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:34.223480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87652a6-1a61-4a23-9244-261d66a9b3ae None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:34.265496 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097acd0-0da6-4382-91d4-63c85f4d82dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:34.266102 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097acd0-0da6-4382-91d4-63c85f4d82dd None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:34.266392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097acd0-0da6-4382-91d4-63c85f4d82dd None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:34.266695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097acd0-0da6-4382-91d4-63c85f4d82dd None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:34.267456 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097acd0-0da6-4382-91d4-63c85f4d82dd None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:34.267766 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097acd0-0da6-4382-91d4-63c85f4d82dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:34.269166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097acd0-0da6-4382-91d4-63c85f4d82dd None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:34.292739 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 162/344] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:34.301295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105822bb-a8ee-44ce-842c-99765b80c8c5 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:34.301590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105822bb-a8ee-44ce-842c-99765b80c8c5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:34.301869 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105822bb-a8ee-44ce-842c-99765b80c8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:34.322888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-105822bb-a8ee-44ce-842c-99765b80c8c5 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:34.324522 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-105822bb-a8ee-44ce-842c-99765b80c8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:34.343082 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:34.371499 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:34.409668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 163/345] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:34.415784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105822bb-a8ee-44ce-842c-99765b80c8c5 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:34.523246 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87652a6-1a61-4a23-9244-261d66a9b3ae None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:34.548051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105822bb-a8ee-44ce-842c-99765b80c8c5 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:34.587229 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635dc199-7ce5-448f-a4dd-dc84b19d209b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:34.588481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dc199-7ce5-448f-a4dd-dc84b19d209b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:34.588913 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dc199-7ce5-448f-a4dd-dc84b19d209b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:34.589336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dc199-7ce5-448f-a4dd-dc84b19d209b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:34.612706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:34.613225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd02140-ad35-421d-8c1a-b7d93c037525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:34.614104 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd02140-ad35-421d-8c1a-b7d93c037525 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:34.614384 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd02140-ad35-421d-8c1a-b7d93c037525 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:34.614682 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd02140-ad35-421d-8c1a-b7d93c037525 None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:34.617022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd02140-ad35-421d-8c1a-b7d93c037525 None admin] RBAC: Authorizing `identity:get_project(project_id=d20d807ab0054a0d994c8b287d262cba)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:34.618909 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd02140-ad35-421d-8c1a-b7d93c037525 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:34.627344 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 183/346] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:34 2022] GET /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba => generated 298 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:34.628027 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd02140-ad35-421d-8c1a-b7d93c037525 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:34.669064 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635dc199-7ce5-448f-a4dd-dc84b19d209b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:34.671392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635dc199-7ce5-448f-a4dd-dc84b19d209b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:34.741033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd02140-ad35-421d-8c1a-b7d93c037525 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:34.773810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:34.784911 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e008a8-ad98-4ddf-aafd-41f5aa8b697b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:34.786118 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 164/347] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:34.786277 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e008a8-ad98-4ddf-aafd-41f5aa8b697b None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:34.786612 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e008a8-ad98-4ddf-aafd-41f5aa8b697b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:34.787110 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e008a8-ad98-4ddf-aafd-41f5aa8b697b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:34.787760 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd76d91-b50d-467f-8387-bb2705d2f361 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:34.788047 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd76d91-b50d-467f-8387-bb2705d2f361 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:34.788319 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd76d91-b50d-467f-8387-bb2705d2f361 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:34.790597 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 165/348] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:34.790597 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd76d91-b50d-467f-8387-bb2705d2f361 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:34.805566 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e008a8-ad98-4ddf-aafd-41f5aa8b697b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:34.817665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-c4e008a8-ad98-4ddf-aafd-41f5aa8b697b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:34.817665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:34.817665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:34.817665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:34.817665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:34.817665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:34.817665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:34.817665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:34.817665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:34.817665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:34.817665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:34.817665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:34.817665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:34.817665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:34.817665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:34.817665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:34.818897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:34.818897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:34.818897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:34.818897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:34.818897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:34.818897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:34.818897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:34.818897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:34.818897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:34.818897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:34.818897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:34.818897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:34.818897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:34.818897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:34.818897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:34.818897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:34.819700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:34.819700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:34.819700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:34.819700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:34.819700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:34.819700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:34.819700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:34.819700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:34.819700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:34.819700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:34.819700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:34.819700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:34.819700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 184/349] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:34.820872 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e2db1-0a75-48cd-ada1-66b66bdb6e2e None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:34.820872 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e2db1-0a75-48cd-ada1-66b66bdb6e2e None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:34.821020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e2db1-0a75-48cd-ada1-66b66bdb6e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:34.845905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-5a7e2db1-0a75-48cd-ada1-66b66bdb6e2e None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:34.847395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-5a7e2db1-0a75-48cd-ada1-66b66bdb6e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:34.875883 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:34.946351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd76d91-b50d-467f-8387-bb2705d2f361 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:34.952162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 185/350] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:34.952930 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e2db1-0a75-48cd-ada1-66b66bdb6e2e None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:34.996272 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec545e-f0f1-4650-9cdf-8e0f9484a560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:34.997072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec545e-f0f1-4650-9cdf-8e0f9484a560 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:34.997354 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec545e-f0f1-4650-9cdf-8e0f9484a560 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:34.997623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec545e-f0f1-4650-9cdf-8e0f9484a560 None admin] PATH_INFO: `/v3/groups/90d4554e0dbf49d39cb31d402accb07f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:35.007401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec545e-f0f1-4650-9cdf-8e0f9484a560 None admin] RBAC: Authorizing `identity:get_group(group_id=90d4554e0dbf49d39cb31d402accb07f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:35.008867 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec545e-f0f1-4650-9cdf-8e0f9484a560 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:35.010804 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 166/351] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 23 11:25:34 2022] GET /identity/v3/groups/90d4554e0dbf49d39cb31d402accb07f => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:35.011468 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec545e-f0f1-4650-9cdf-8e0f9484a560 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:35.074678 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e2db1-0a75-48cd-ada1-66b66bdb6e2e None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:35.101863 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:35.115099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73c5ae-ab80-448a-8bc0-38b895c7b175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:35.115971 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73c5ae-ab80-448a-8bc0-38b895c7b175 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:35.116260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73c5ae-ab80-448a-8bc0-38b895c7b175 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:35.116674 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73c5ae-ab80-448a-8bc0-38b895c7b175 None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:35.118248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73c5ae-ab80-448a-8bc0-38b895c7b175 None admin] RBAC: Authorizing `identity:get_project(project_id=d20d807ab0054a0d994c8b287d262cba)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:35.119775 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73c5ae-ab80-448a-8bc0-38b895c7b175 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:35.123155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 186/352] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:34 2022] GET /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:35.127136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9722e9-c0a9-4009-9af0-5b517a1e352e None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:35.127407 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9722e9-c0a9-4009-9af0-5b517a1e352e None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:35.127692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9722e9-c0a9-4009-9af0-5b517a1e352e None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:35.129137 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 187/353] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:35.129806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9722e9-c0a9-4009-9af0-5b517a1e352e None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:35.131553 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec545e-f0f1-4650-9cdf-8e0f9484a560 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:35.150836 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:35.178875 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573133d6-6663-45a5-a91b-b4a81defdd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:35.179616 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573133d6-6663-45a5-a91b-b4a81defdd71 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:35.179902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573133d6-6663-45a5-a91b-b4a81defdd71 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:35.180230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573133d6-6663-45a5-a91b-b4a81defdd71 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:35.180989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573133d6-6663-45a5-a91b-b4a81defdd71 None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:35.181344 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573133d6-6663-45a5-a91b-b4a81defdd71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:35.183594 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573133d6-6663-45a5-a91b-b4a81defdd71 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:35.193628 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 167/354] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:35.196121 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4e930-fe53-4c55-838d-b5f1ac257b55 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:35.196121 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4e930-fe53-4c55-838d-b5f1ac257b55 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:35.196121 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4e930-fe53-4c55-838d-b5f1ac257b55 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:35.197503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 168/355] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:35.199707 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4e930-fe53-4c55-838d-b5f1ac257b55 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:35.266686 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9722e9-c0a9-4009-9af0-5b517a1e352e None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:35.315724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d882cb-8314-41c7-8e00-e2b47b7dcee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:35.316680 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d882cb-8314-41c7-8e00-e2b47b7dcee0 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:35.317086 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d882cb-8314-41c7-8e00-e2b47b7dcee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:35.317523 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d882cb-8314-41c7-8e00-e2b47b7dcee0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:35.319080 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4e930-fe53-4c55-838d-b5f1ac257b55 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:35.333844 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d882cb-8314-41c7-8e00-e2b47b7dcee0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:35.347152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-a6d882cb-8314-41c7-8e00-e2b47b7dcee0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:35.347152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:35.347152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:35.347152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:35.347152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:35.347152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:35.347152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:35.347152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:35.347152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:35.347152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:35.347152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:35.347152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:35.347152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:35.347152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:35.347152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:35.347152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:35.348279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:35.348279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:35.348279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:35.348279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:35.348279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:35.348279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:35.348279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:35.348279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:35.348279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:35.348279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:35.348279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:35.348279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:35.348279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:35.348279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:35.348279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:35.348279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:35.349241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:35.349241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:35.349241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:35.349241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:35.349241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:35.349241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:35.349241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:35.349241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:35.349241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:35.349241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:35.349241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:35.349241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:35.349241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 188/356] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:25:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:35.350068 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d882cb-8314-41c7-8e00-e2b47b7dcee0 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:35.364535 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a654b-f5ad-4bc3-952b-b42f35162a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:35.365201 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a654b-f5ad-4bc3-952b-b42f35162a21 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:35.365478 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a654b-f5ad-4bc3-952b-b42f35162a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:35.365746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a654b-f5ad-4bc3-952b-b42f35162a21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:35.367101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a654b-f5ad-4bc3-952b-b42f35162a21 None admin] RBAC: Adding query filter params (group.id=90d4554e0dbf49d39cb31d402accb07f, role.id=25e99b2a08914aa0ab18cc4955f25364, scope.project.id=d20d807ab0054a0d994c8b287d262cba) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:35.367101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a654b-f5ad-4bc3-952b-b42f35162a21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:35.368329 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a654b-f5ad-4bc3-952b-b42f35162a21 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:35.377162 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 169/357] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jun 23 11:25:35 2022] GET /identity/v3/role_assignments?group.id=90d4554e0dbf49d39cb31d402accb07f&scope.project.id=d20d807ab0054a0d994c8b287d262cba&role.id=25e99b2a08914aa0ab18cc4955f25364 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:35.378939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb318d8-9b28-4651-b2ce-34d49b7163c4 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:35.379426 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb318d8-9b28-4651-b2ce-34d49b7163c4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:35.379891 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb318d8-9b28-4651-b2ce-34d49b7163c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:35.399681 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-beb318d8-9b28-4651-b2ce-34d49b7163c4 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:35.401840 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-beb318d8-9b28-4651-b2ce-34d49b7163c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:35.439631 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:35.455778 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:35.516170 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 170/358] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:35.521865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d882cb-8314-41c7-8e00-e2b47b7dcee0 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:35.523740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb318d8-9b28-4651-b2ce-34d49b7163c4 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:35.564763 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433e81b-c454-42ef-8db6-e1df91305e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:35.565646 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433e81b-c454-42ef-8db6-e1df91305e30 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:35.565956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433e81b-c454-42ef-8db6-e1df91305e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:35.566233 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433e81b-c454-42ef-8db6-e1df91305e30 None admin] PATH_INFO: `/v3/groups/90d4554e0dbf49d39cb31d402accb07f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:35.568865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433e81b-c454-42ef-8db6-e1df91305e30 None admin] RBAC: Authorizing `identity:get_group(group_id=90d4554e0dbf49d39cb31d402accb07f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:35.570343 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433e81b-c454-42ef-8db6-e1df91305e30 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:35.572017 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 189/359] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 23 11:25:35 2022] GET /identity/v3/groups/90d4554e0dbf49d39cb31d402accb07f => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:35.573191 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894e96c-31a5-449d-95bf-8d37889b960f None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:35.573491 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894e96c-31a5-449d-95bf-8d37889b960f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:35.573753 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894e96c-31a5-449d-95bf-8d37889b960f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:35.596229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-a894e96c-31a5-449d-95bf-8d37889b960f None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:35.596229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a894e96c-31a5-449d-95bf-8d37889b960f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:35.663142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb318d8-9b28-4651-b2ce-34d49b7163c4 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:35.680644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 190/360] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:35.682938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894e96c-31a5-449d-95bf-8d37889b960f None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:35.709893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c79eb-ec03-4029-96e3-76ec573b9210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:35.710580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c79eb-ec03-4029-96e3-76ec573b9210 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:35.710856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c79eb-ec03-4029-96e3-76ec573b9210 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:35.711574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c79eb-ec03-4029-96e3-76ec573b9210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:35.746918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:35.760312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:35.799183 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c79eb-ec03-4029-96e3-76ec573b9210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:35.800569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c79eb-ec03-4029-96e3-76ec573b9210 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:35.813473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894e96c-31a5-449d-95bf-8d37889b960f None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:35.856333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4772244-2049-4c09-aeb4-e26c58200a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:35.857105 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4772244-2049-4c09-aeb4-e26c58200a72 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:35.857397 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4772244-2049-4c09-aeb4-e26c58200a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:35.857673 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4772244-2049-4c09-aeb4-e26c58200a72 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:35.858461 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4772244-2049-4c09-aeb4-e26c58200a72 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:35.858817 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4772244-2049-4c09-aeb4-e26c58200a72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:35.860197 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4772244-2049-4c09-aeb4-e26c58200a72 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:35.870268 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 191/361] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:25:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:35.873925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4772244-2049-4c09-aeb4-e26c58200a72 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:35.943704 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 171/362] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:35.944517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c79eb-ec03-4029-96e3-76ec573b9210 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:36.011934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4772244-2049-4c09-aeb4-e26c58200a72 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:36.028531 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:36.036286 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:36.062697 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5608d10-6a13-44fe-9304-76be103659ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:36.062697 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5608d10-6a13-44fe-9304-76be103659ed None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:36.062989 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5608d10-6a13-44fe-9304-76be103659ed None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:36.063233 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5608d10-6a13-44fe-9304-76be103659ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:36.077481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c79eb-ec03-4029-96e3-76ec573b9210 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:36.079676 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5608d10-6a13-44fe-9304-76be103659ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:36.089322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-e5608d10-6a13-44fe-9304-76be103659ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:25:36.089322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:36.089322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:36.089322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:36.089322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:36.089322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:36.089322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:36.089322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:36.089322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:36.089322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:36.089322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:36.089322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:36.089322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:36.089322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:36.089322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:36.089322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:36.090302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:36.090302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:36.090302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:36.090302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:36.090302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:36.090302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:36.090302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:36.090302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:36.090302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:36.090302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:36.090302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:36.090302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:36.090302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:36.090302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:36.090302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:36.090302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:36.091180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:36.091180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:36.091180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:36.091180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:36.091180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:36.091180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:36.091180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:36.091180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:36.091180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:36.091180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:36.091180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:25:36.091180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:36.091817 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 192/363] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:25:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:36.092396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5608d10-6a13-44fe-9304-76be103659ed None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:36.133762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f2769-fd08-4846-9a7d-6d884a4f6d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:36.134418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f2769-fd08-4846-9a7d-6d884a4f6d94 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:36.134732 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f2769-fd08-4846-9a7d-6d884a4f6d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:36.135013 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f2769-fd08-4846-9a7d-6d884a4f6d94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:36.135854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f2769-fd08-4846-9a7d-6d884a4f6d94 None admin] RBAC: Adding query filter params (group.id=90d4554e0dbf49d39cb31d402accb07f, role.id=75f08cd0671f436c9f968976aa8baa7f, scope.project.id=d20d807ab0054a0d994c8b287d262cba) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:36.136171 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f2769-fd08-4846-9a7d-6d884a4f6d94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:36.137623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f2769-fd08-4846-9a7d-6d884a4f6d94 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:36.150563 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 172/364] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jun 23 11:25:35 2022] GET /identity/v3/role_assignments?group.id=90d4554e0dbf49d39cb31d402accb07f&scope.project.id=d20d807ab0054a0d994c8b287d262cba&role.id=75f08cd0671f436c9f968976aa8baa7f => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:36.150563 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f2769-fd08-4846-9a7d-6d884a4f6d94 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:36.247801 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5608d10-6a13-44fe-9304-76be103659ed None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:36.282459 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f2769-fd08-4846-9a7d-6d884a4f6d94 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:36.289013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a9295-504d-4300-a636-8b897c40a208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:36.289834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a9295-504d-4300-a636-8b897c40a208 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:36.290118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a9295-504d-4300-a636-8b897c40a208 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:36.290394 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a9295-504d-4300-a636-8b897c40a208 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:36.303308 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0a9295-504d-4300-a636-8b897c40a208 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:36.313800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-5e0a9295-504d-4300-a636-8b897c40a208 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:25:36.313800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:36.313800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:36.313800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:36.313800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:36.313800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:36.313800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:36.313800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:36.313800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:36.313800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:36.313800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:36.313800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:36.313800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:36.313800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:36.313800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:36.313800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:36.314785 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:36.314785 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:36.314785 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:36.314785 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:36.314785 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:36.314785 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:36.314785 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:36.314785 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:36.314785 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:36.314785 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:36.314785 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:36.314785 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:36.314785 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:36.314785 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:36.314785 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:36.314785 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:36.316020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:36.316020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:36.316020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:36.316020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:36.316020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:36.316020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:36.316020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:36.316020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:36.316020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:36.316020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:36.316020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:25:36.316020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:36.316020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 193/365] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:36.316706 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a9295-504d-4300-a636-8b897c40a208 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:36.328228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed2f4e-897e-4bca-96c8-529d1be7bcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:36.328915 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed2f4e-897e-4bca-96c8-529d1be7bcf4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:36.329202 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed2f4e-897e-4bca-96c8-529d1be7bcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:36.329481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed2f4e-897e-4bca-96c8-529d1be7bcf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:36.342024 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:36.359721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:36.419263 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efed2f4e-897e-4bca-96c8-529d1be7bcf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:36.421447 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efed2f4e-897e-4bca-96c8-529d1be7bcf4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:36.444795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a9295-504d-4300-a636-8b897c40a208 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:36.506666 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32b8e4-f42d-4e8e-8e83-c4828ea77088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:36.507919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32b8e4-f42d-4e8e-8e83-c4828ea77088 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:36.508415 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32b8e4-f42d-4e8e-8e83-c4828ea77088 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:36.508901 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32b8e4-f42d-4e8e-8e83-c4828ea77088 None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:36.510362 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32b8e4-f42d-4e8e-8e83-c4828ea77088 None admin] RBAC: Authorizing `identity:get_project(project_id=b05339ac8c27421a9a1921726dcbdc7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:36.512070 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32b8e4-f42d-4e8e-8e83-c4828ea77088 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:36.514385 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 194/366] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:36 2022] GET /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:36.515582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32b8e4-f42d-4e8e-8e83-c4828ea77088 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:36.559111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 173/367] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:36.562793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed2f4e-897e-4bca-96c8-529d1be7bcf4 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:36.601527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:36.658153 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:36.664055 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32b8e4-f42d-4e8e-8e83-c4828ea77088 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:36.681030 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed2f4e-897e-4bca-96c8-529d1be7bcf4 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:36.713385 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2020f0a-a39a-465f-baeb-6a246dce688d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:36.714121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2020f0a-a39a-465f-baeb-6a246dce688d None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:36.714415 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2020f0a-a39a-465f-baeb-6a246dce688d None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:36.714764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2020f0a-a39a-465f-baeb-6a246dce688d None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:36.715787 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2020f0a-a39a-465f-baeb-6a246dce688d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:36.716130 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2020f0a-a39a-465f-baeb-6a246dce688d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:36.719020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2020f0a-a39a-465f-baeb-6a246dce688d None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:36.722279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d6a55-25cd-44a0-94bc-363fb0ba38c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:36.723261 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d6a55-25cd-44a0-94bc-363fb0ba38c1 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:36.723748 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d6a55-25cd-44a0-94bc-363fb0ba38c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:36.724237 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d6a55-25cd-44a0-94bc-363fb0ba38c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:36.725199 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d6a55-25cd-44a0-94bc-363fb0ba38c1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:36.725718 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d6a55-25cd-44a0-94bc-363fb0ba38c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:36.727360 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d6a55-25cd-44a0-94bc-363fb0ba38c1 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:36.728339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 195/368] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:36.729157 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2020f0a-a39a-465f-baeb-6a246dce688d None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:36.738356 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 174/369] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:36.739581 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d6a55-25cd-44a0-94bc-363fb0ba38c1 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:36.861706 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2020f0a-a39a-465f-baeb-6a246dce688d None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:36.871958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:36.875380 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d6a55-25cd-44a0-94bc-363fb0ba38c1 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:36.909150 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db859f7f-e543-44fc-a77b-fd3525c8c20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:36.910394 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db859f7f-e543-44fc-a77b-fd3525c8c20e None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:36.910920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db859f7f-e543-44fc-a77b-fd3525c8c20e None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:36.911434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db859f7f-e543-44fc-a77b-fd3525c8c20e None admin] PATH_INFO: `/v3/groups/90d4554e0dbf49d39cb31d402accb07f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:36.912839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db859f7f-e543-44fc-a77b-fd3525c8c20e None admin] RBAC: Authorizing `identity:get_group(group_id=90d4554e0dbf49d39cb31d402accb07f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:36.913150 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a13b1-ce11-49cd-9f41-5429ca4b1564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:36.914021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a13b1-ce11-49cd-9f41-5429ca4b1564 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:36.914304 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a13b1-ce11-49cd-9f41-5429ca4b1564 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:36.914630 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a13b1-ce11-49cd-9f41-5429ca4b1564 None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:36.915492 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db859f7f-e543-44fc-a77b-fd3525c8c20e None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:36.915937 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a13b1-ce11-49cd-9f41-5429ca4b1564 None admin] RBAC: Authorizing `identity:get_project(project_id=b05339ac8c27421a9a1921726dcbdc7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:36.917392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a13b1-ce11-49cd-9f41-5429ca4b1564 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:36.917815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 196/370] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 23 11:25:36 2022] GET /identity/v3/groups/90d4554e0dbf49d39cb31d402accb07f => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:36.918940 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db859f7f-e543-44fc-a77b-fd3525c8c20e None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:36.919371 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 175/371] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:36 2022] GET /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:36.920261 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a13b1-ce11-49cd-9f41-5429ca4b1564 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:36.953760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:37.029482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a13b1-ce11-49cd-9f41-5429ca4b1564 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:37.034716 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db859f7f-e543-44fc-a77b-fd3525c8c20e None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:37.070395 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b461209-8b06-486c-83f7-a077c0030da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:37.071397 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b461209-8b06-486c-83f7-a077c0030da6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:37.071768 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b461209-8b06-486c-83f7-a077c0030da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:37.072054 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b461209-8b06-486c-83f7-a077c0030da6 None admin] PATH_INFO: `/v3/users/1913d1b97f0f468f9686c3ac9537b446` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:37.074621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b461209-8b06-486c-83f7-a077c0030da6 None admin] RBAC: Authorizing `identity:get_user(user_id=1913d1b97f0f468f9686c3ac9537b446)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:37.074980 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a73923-9b33-4786-a556-32369d1c765d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:37.075850 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a73923-9b33-4786-a556-32369d1c765d None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:37.076258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b461209-8b06-486c-83f7-a077c0030da6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:37.076557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a73923-9b33-4786-a556-32369d1c765d None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:37.077599 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a73923-9b33-4786-a556-32369d1c765d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:37.078825 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 176/372] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:25:36 2022] GET /identity/v3/users/1913d1b97f0f468f9686c3ac9537b446 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:37.079597 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b461209-8b06-486c-83f7-a077c0030da6 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:37.119111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:37.153167 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a73923-9b33-4786-a556-32369d1c765d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:37.155183 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a73923-9b33-4786-a556-32369d1c765d None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:37.188871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:37.194315 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b461209-8b06-486c-83f7-a077c0030da6 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:37.233361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ff1f6-1618-4dcd-ae90-a0f4bb3c1510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:37.234134 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ff1f6-1618-4dcd-ae90-a0f4bb3c1510 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:37.234390 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ff1f6-1618-4dcd-ae90-a0f4bb3c1510 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:37.234700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ff1f6-1618-4dcd-ae90-a0f4bb3c1510 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:37.235648 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ff1f6-1618-4dcd-ae90-a0f4bb3c1510 None admin] RBAC: Adding query filter params (group.id=90d4554e0dbf49d39cb31d402accb07f, role.id=75f08cd0671f436c9f968976aa8baa7f, scope.project.id=b05339ac8c27421a9a1921726dcbdc7f) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:37.235966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ff1f6-1618-4dcd-ae90-a0f4bb3c1510 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:37.238278 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ff1f6-1618-4dcd-ae90-a0f4bb3c1510 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:37.243854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 177/373] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jun 23 11:25:37 2022] GET /identity/v3/role_assignments?group.id=90d4554e0dbf49d39cb31d402accb07f&scope.project.id=b05339ac8c27421a9a1921726dcbdc7f&role.id=75f08cd0671f436c9f968976aa8baa7f => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:37.244674 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ff1f6-1618-4dcd-ae90-a0f4bb3c1510 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:37.281686 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 197/374] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:37.287865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a73923-9b33-4786-a556-32369d1c765d None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:37.358655 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:37.365777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ff1f6-1618-4dcd-ae90-a0f4bb3c1510 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:37.403661 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231783b-280b-4c4b-90d5-9038472377b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:37.404912 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231783b-280b-4c4b-90d5-9038472377b7 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:37.405338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231783b-280b-4c4b-90d5-9038472377b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:37.405709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231783b-280b-4c4b-90d5-9038472377b7 None admin] PATH_INFO: `/v3/groups/90d4554e0dbf49d39cb31d402accb07f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:37.407029 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f231783b-280b-4c4b-90d5-9038472377b7 None admin] RBAC: Authorizing `identity:get_group(group_id=90d4554e0dbf49d39cb31d402accb07f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:37.408589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f231783b-280b-4c4b-90d5-9038472377b7 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:37.411333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 178/375] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 23 11:25:37 2022] GET /identity/v3/groups/90d4554e0dbf49d39cb31d402accb07f => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:37.416283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231783b-280b-4c4b-90d5-9038472377b7 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:37.442405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a73923-9b33-4786-a556-32369d1c765d None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:37.486373 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:37.504217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f235e9-5807-48a1-993c-a3ee0d66209c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:37.504217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f235e9-5807-48a1-993c-a3ee0d66209c None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:37.504217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f235e9-5807-48a1-993c-a3ee0d66209c None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:37.504638 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f235e9-5807-48a1-993c-a3ee0d66209c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:37.505683 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f235e9-5807-48a1-993c-a3ee0d66209c None admin] RBAC: Adding query filter params (role.id=c8e4f6d521b04ad39661b74575a8b9d3, user.id=1913d1b97f0f468f9686c3ac9537b446, scope.system=all) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:37.506078 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f235e9-5807-48a1-993c-a3ee0d66209c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:37.511838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f235e9-5807-48a1-993c-a3ee0d66209c None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:37.532663 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 198/376] 10.222.0.13 () {56 vars in 1308 bytes} [Thu Jun 23 11:25:37 2022] GET /identity/v3/role_assignments?user.id=1913d1b97f0f468f9686c3ac9537b446&scope.system=all&role.id=c8e4f6d521b04ad39661b74575a8b9d3 => generated 510 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:37.538565 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f235e9-5807-48a1-993c-a3ee0d66209c None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:37.570237 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231783b-280b-4c4b-90d5-9038472377b7 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:37.631429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96e116-c01c-4b78-adb0-9dd19c7eaba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:37.632733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96e116-c01c-4b78-adb0-9dd19c7eaba0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:37.633305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96e116-c01c-4b78-adb0-9dd19c7eaba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:37.633768 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96e116-c01c-4b78-adb0-9dd19c7eaba0 None admin] PATH_INFO: `/v3/projects/237fd59b9b4946f0a2bdfcf73db9274e` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:37.635462 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96e116-c01c-4b78-adb0-9dd19c7eaba0 None admin] RBAC: Authorizing `identity:get_project(project_id=237fd59b9b4946f0a2bdfcf73db9274e)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:37.637180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96e116-c01c-4b78-adb0-9dd19c7eaba0 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:37.639702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 179/377] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:37 2022] GET /identity/v3/projects/237fd59b9b4946f0a2bdfcf73db9274e => generated 344 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:37.641434 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fe352-e5bf-435a-b61e-c1987eb98b70 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:37.641983 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fe352-e5bf-435a-b61e-c1987eb98b70 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:37.642444 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fe352-e5bf-435a-b61e-c1987eb98b70 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:37.644046 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 180/378] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:37.645014 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99fe352-e5bf-435a-b61e-c1987eb98b70 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:37.690926 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:37.745459 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f235e9-5807-48a1-993c-a3ee0d66209c None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:37.777028 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99fe352-e5bf-435a-b61e-c1987eb98b70 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:37.792194 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea1ccd-36c9-48a8-8969-de773f71c2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:37.792947 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea1ccd-36c9-48a8-8969-de773f71c2fe None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:37.793298 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea1ccd-36c9-48a8-8969-de773f71c2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:37.793643 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea1ccd-36c9-48a8-8969-de773f71c2fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:37.794694 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea1ccd-36c9-48a8-8969-de773f71c2fe None admin] RBAC: Adding query filter params (group.id=90d4554e0dbf49d39cb31d402accb07f, role.id=25e99b2a08914aa0ab18cc4955f25364, scope.project.id=b05339ac8c27421a9a1921726dcbdc7f) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:37.795182 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea1ccd-36c9-48a8-8969-de773f71c2fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:37.797427 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea1ccd-36c9-48a8-8969-de773f71c2fe None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:37.810178 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 199/379] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jun 23 11:25:37 2022] GET /identity/v3/role_assignments?group.id=90d4554e0dbf49d39cb31d402accb07f&scope.project.id=b05339ac8c27421a9a1921726dcbdc7f&role.id=25e99b2a08914aa0ab18cc4955f25364 => generated 260 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:37.810178 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9f42d-a816-42e8-a9e0-219912ed5123 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:37.810178 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9f42d-a816-42e8-a9e0-219912ed5123 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:37.810178 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9f42d-a816-42e8-a9e0-219912ed5123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:37.833027 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-56e9f42d-a816-42e8-a9e0-219912ed5123 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:37.834263 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-56e9f42d-a816-42e8-a9e0-219912ed5123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:37.844954 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08280760-0230-4e37-b480-2f1160e892d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:37.845838 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08280760-0230-4e37-b480-2f1160e892d7 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:37.846119 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08280760-0230-4e37-b480-2f1160e892d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:37.846392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08280760-0230-4e37-b480-2f1160e892d7 None admin] PATH_INFO: `/v3/groups/2a271c10c7e1469bbae9bdea7c6cc43f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:37.859152 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08280760-0230-4e37-b480-2f1160e892d7 None admin] RBAC: Authorizing `identity:get_group(group_id=2a271c10c7e1469bbae9bdea7c6cc43f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:37.860636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08280760-0230-4e37-b480-2f1160e892d7 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:37.864282 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 181/380] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 23 11:25:37 2022] GET /identity/v3/groups/2a271c10c7e1469bbae9bdea7c6cc43f => generated 229 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:37.872765 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08280760-0230-4e37-b480-2f1160e892d7 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:37.938813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:37.983263 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 200/381] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:38.019033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9f42d-a816-42e8-a9e0-219912ed5123 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:38.052027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:38.063418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08280760-0230-4e37-b480-2f1160e892d7 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:38.114268 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfe2d5-f3b1-477e-8250-c83433b63383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:38.115096 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfe2d5-f3b1-477e-8250-c83433b63383 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:38.115382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfe2d5-f3b1-477e-8250-c83433b63383 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:38.115654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfe2d5-f3b1-477e-8250-c83433b63383 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:38.116496 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecfe2d5-f3b1-477e-8250-c83433b63383 None admin] RBAC: Adding query filter params (group.id=2a271c10c7e1469bbae9bdea7c6cc43f, role.id=c9e006de0d6c4812ab29d778e9c8a830, scope.project.id=237fd59b9b4946f0a2bdfcf73db9274e) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:38.116805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecfe2d5-f3b1-477e-8250-c83433b63383 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:38.120222 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecfe2d5-f3b1-477e-8250-c83433b63383 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:38.134257 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 182/382] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jun 23 11:25:37 2022] GET /identity/v3/role_assignments?group.id=2a271c10c7e1469bbae9bdea7c6cc43f&scope.project.id=237fd59b9b4946f0a2bdfcf73db9274e&role.id=c9e006de0d6c4812ab29d778e9c8a830 => generated 260 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:38.174128 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9f42d-a816-42e8-a9e0-219912ed5123 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:38.230198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a803538-3285-4203-bc79-049fd07a86c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:38.231068 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a803538-3285-4203-bc79-049fd07a86c6 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:38.231354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a803538-3285-4203-bc79-049fd07a86c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:38.231617 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a803538-3285-4203-bc79-049fd07a86c6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:38.248620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a803538-3285-4203-bc79-049fd07a86c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:38.257426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-0a803538-3285-4203-bc79-049fd07a86c6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:38.257426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:38.257426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:38.257426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:38.257426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:38.257426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:38.257426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:38.257426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:38.257426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:38.257426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:38.257426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:38.257426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:38.257426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:38.257426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:38.257426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:38.257426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:38.258379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:38.258379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:38.258379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:38.258379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:38.258379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:38.258379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:38.258379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:38.258379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:38.258379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:38.258379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:38.258379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:38.258379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:38.258379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:38.258379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:38.258379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:38.258379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:38.259212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:38.259212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:38.259212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:38.259212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:38.259212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:38.259212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:38.259212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:38.259212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:38.259212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:38.259212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:38.259212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:38.259212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:38.263186 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 201/383] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:25:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:25:38.269758 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfe2d5-f3b1-477e-8250-c83433b63383 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:38.409692 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfe2d5-f3b1-477e-8250-c83433b63383 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:38.455564 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0dfd4-1f01-48ae-8511-7431ad63828c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:38.456243 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0dfd4-1f01-48ae-8511-7431ad63828c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:38.456763 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0dfd4-1f01-48ae-8511-7431ad63828c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:38.457033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0dfd4-1f01-48ae-8511-7431ad63828c None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:38.457793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0dfd4-1f01-48ae-8511-7431ad63828c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:38.458100 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0dfd4-1f01-48ae-8511-7431ad63828c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:38.459796 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0dfd4-1f01-48ae-8511-7431ad63828c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:38.470023 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 183/384] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:25:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:38.475479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a803538-3285-4203-bc79-049fd07a86c6 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:38.527038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:38.584442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a803538-3285-4203-bc79-049fd07a86c6 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:38.621957 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87365bdc-e5bf-4ada-8a01-a3d42f57fe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:38.622859 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87365bdc-e5bf-4ada-8a01-a3d42f57fe8c None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:38.623140 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87365bdc-e5bf-4ada-8a01-a3d42f57fe8c None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:38.623437 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87365bdc-e5bf-4ada-8a01-a3d42f57fe8c None admin] PATH_INFO: `/v3/groups/90d4554e0dbf49d39cb31d402accb07f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:38.625351 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87365bdc-e5bf-4ada-8a01-a3d42f57fe8c None admin] RBAC: Authorizing `identity:get_group(group_id=90d4554e0dbf49d39cb31d402accb07f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:38.626818 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87365bdc-e5bf-4ada-8a01-a3d42f57fe8c None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:38.628797 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 202/385] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 23 11:25:38 2022] GET /identity/v3/groups/90d4554e0dbf49d39cb31d402accb07f => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:38.640227 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0dfd4-1f01-48ae-8511-7431ad63828c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:38.674442 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:38.763920 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0dfd4-1f01-48ae-8511-7431ad63828c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:38.803838 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989c8d0-2854-42c9-9c50-869d830654c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:38.804625 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989c8d0-2854-42c9-9c50-869d830654c5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:38.804902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989c8d0-2854-42c9-9c50-869d830654c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:38.805177 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989c8d0-2854-42c9-9c50-869d830654c5 None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:38.806368 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989c8d0-2854-42c9-9c50-869d830654c5 None admin] RBAC: Authorizing `identity:get_project(project_id=d20d807ab0054a0d994c8b287d262cba)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:38.807901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989c8d0-2854-42c9-9c50-869d830654c5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:38.809591 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 184/386] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:38 2022] GET /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:38.816028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87365bdc-e5bf-4ada-8a01-a3d42f57fe8c None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:38.832695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa89fdd-cefb-4594-ba7d-2be597eb3f55 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:38.833011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa89fdd-cefb-4594-ba7d-2be597eb3f55 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:38.833248 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa89fdd-cefb-4594-ba7d-2be597eb3f55 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:38.834890 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 185/387] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:38.845126 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c823a5-69ef-46fe-ab86-4a697276d70a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:38.845411 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c823a5-69ef-46fe-ab86-4a697276d70a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:38.845684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c823a5-69ef-46fe-ab86-4a697276d70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:38.866918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-06c823a5-69ef-46fe-ab86-4a697276d70a None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:38.868384 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-06c823a5-69ef-46fe-ab86-4a697276d70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:38.991774 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87365bdc-e5bf-4ada-8a01-a3d42f57fe8c None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:39.001343 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:39.024091 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 186/388] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:39.038989 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabbbc7-3f3d-43fa-8ba5-f1aa1ba4c8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:39.039917 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabbbc7-3f3d-43fa-8ba5-f1aa1ba4c8bf None admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:39.040207 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabbbc7-3f3d-43fa-8ba5-f1aa1ba4c8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:39.040488 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabbbc7-3f3d-43fa-8ba5-f1aa1ba4c8bf None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba/groups/90d4554e0dbf49d39cb31d402accb07f/roles/25e99b2a08914aa0ab18cc4955f25364` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:39.046112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceabbbc7-3f3d-43fa-8ba5-f1aa1ba4c8bf None admin] RBAC: Authorizing `identity:create_grant(project_id=d20d807ab0054a0d994c8b287d262cba, group_id=90d4554e0dbf49d39cb31d402accb07f, role_id=25e99b2a08914aa0ab18cc4955f25364)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:39.048232 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceabbbc7-3f3d-43fa-8ba5-f1aa1ba4c8bf None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:39.063249 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c823a5-69ef-46fe-ab86-4a697276d70a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:39.090949 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:39.126158 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 203/389] 10.222.0.13 () {58 vars in 1442 bytes} [Thu Jun 23 11:25:38 2022] PUT /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba/groups/90d4554e0dbf49d39cb31d402accb07f/roles/25e99b2a08914aa0ab18cc4955f25364 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:25:39.187390 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c823a5-69ef-46fe-ab86-4a697276d70a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:39.234473 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758e6fb-e18e-48c9-9559-b3cd28d76bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:39.235321 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758e6fb-e18e-48c9-9559-b3cd28d76bfa None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:39.235750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758e6fb-e18e-48c9-9559-b3cd28d76bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:39.236032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758e6fb-e18e-48c9-9559-b3cd28d76bfa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:39.253163 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758e6fb-e18e-48c9-9559-b3cd28d76bfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:39.264784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-1758e6fb-e18e-48c9-9559-b3cd28d76bfa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:39.264784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:39.264784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:39.264784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:39.264784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:39.264784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:39.264784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:39.264784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:39.264784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:39.264784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:39.264784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:39.264784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:39.264784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:39.264784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:39.264784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:39.264784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:39.265734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:39.265734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:39.265734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:39.265734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:39.265734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:39.265734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:39.265734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:39.265734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:39.265734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:39.265734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:39.265734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:39.265734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:39.265734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:39.265734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:39.265734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:39.265734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:39.266556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:39.266556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:39.266556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:39.266556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:39.266556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:39.266556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:39.266556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:39.266556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:39.266556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:39.266556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:39.266556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:39.266556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:39.266556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 187/390] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:39.272383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabbbc7-3f3d-43fa-8ba5-f1aa1ba4c8bf None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:39.387174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabbbc7-3f3d-43fa-8ba5-f1aa1ba4c8bf None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:39.426144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0330ace-3aea-4435-b066-a16bd1c13b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:39.426849 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0330ace-3aea-4435-b066-a16bd1c13b81 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:39.427137 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0330ace-3aea-4435-b066-a16bd1c13b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:39.427416 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0330ace-3aea-4435-b066-a16bd1c13b81 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:39.428164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0330ace-3aea-4435-b066-a16bd1c13b81 None admin] RBAC: Adding query filter params (name=member) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:39.428479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0330ace-3aea-4435-b066-a16bd1c13b81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:39.429921 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0330ace-3aea-4435-b066-a16bd1c13b81 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:39.446592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 204/391] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:39.452093 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758e6fb-e18e-48c9-9559-b3cd28d76bfa None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:39.569960 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:39.589208 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758e6fb-e18e-48c9-9559-b3cd28d76bfa None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:39.630502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a5a4a-9c05-4b77-a106-62813fff5b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:39.631395 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a5a4a-9c05-4b77-a106-62813fff5b9c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:39.631677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a5a4a-9c05-4b77-a106-62813fff5b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:39.631950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a5a4a-9c05-4b77-a106-62813fff5b9c None admin] PATH_INFO: `/v3/groups/90d4554e0dbf49d39cb31d402accb07f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:39.638855 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a5a4a-9c05-4b77-a106-62813fff5b9c None admin] RBAC: Authorizing `identity:get_group(group_id=90d4554e0dbf49d39cb31d402accb07f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:39.640794 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a5a4a-9c05-4b77-a106-62813fff5b9c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:39.642732 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 188/392] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 23 11:25:39 2022] GET /identity/v3/groups/90d4554e0dbf49d39cb31d402accb07f => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:39.651708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0330ace-3aea-4435-b066-a16bd1c13b81 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:39.702628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:39.799375 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0330ace-3aea-4435-b066-a16bd1c13b81 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:39.836187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f01726-e18d-44f6-9ce1-890dad42915b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:39.837150 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f01726-e18d-44f6-9ce1-890dad42915b None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:39.837447 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f01726-e18d-44f6-9ce1-890dad42915b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:39.837726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f01726-e18d-44f6-9ce1-890dad42915b None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:39.839126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f01726-e18d-44f6-9ce1-890dad42915b None admin] RBAC: Authorizing `identity:get_project(project_id=d20d807ab0054a0d994c8b287d262cba)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:39.840569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f01726-e18d-44f6-9ce1-890dad42915b None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:39.846728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 205/393] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:39 2022] GET /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:39.848687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a5a4a-9c05-4b77-a106-62813fff5b9c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:39.973862 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a5a4a-9c05-4b77-a106-62813fff5b9c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:40.008881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8aee4f-6cd7-4f5f-9134-e428f474183d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:40.009771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8aee4f-6cd7-4f5f-9134-e428f474183d None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:40.010049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8aee4f-6cd7-4f5f-9134-e428f474183d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:40.010322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8aee4f-6cd7-4f5f-9134-e428f474183d None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba/groups/90d4554e0dbf49d39cb31d402accb07f/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:40.012306 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8aee4f-6cd7-4f5f-9134-e428f474183d None admin] RBAC: Authorizing `identity:create_grant(project_id=d20d807ab0054a0d994c8b287d262cba, group_id=90d4554e0dbf49d39cb31d402accb07f, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:40.013757 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8aee4f-6cd7-4f5f-9134-e428f474183d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:40.056395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007e937-b67e-4c0f-9be7-e46a47df8fef None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:40.056994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007e937-b67e-4c0f-9be7-e46a47df8fef None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:40.057452 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007e937-b67e-4c0f-9be7-e46a47df8fef None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:40.059315 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 206/394] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:40.065207 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206c7f5-d5bf-4f9c-9601-163da85ba227 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:40.065683 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206c7f5-d5bf-4f9c-9601-163da85ba227 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:40.066128 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206c7f5-d5bf-4f9c-9601-163da85ba227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:40.084826 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-b206c7f5-d5bf-4f9c-9601-163da85ba227 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:40.086289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b206c7f5-d5bf-4f9c-9601-163da85ba227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:40.102565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 189/395] 10.222.0.13 () {58 vars in 1442 bytes} [Thu Jun 23 11:25:39 2022] PUT /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba/groups/90d4554e0dbf49d39cb31d402accb07f/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:25:40.182691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 207/396] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:25:40.202448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8aee4f-6cd7-4f5f-9134-e428f474183d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:40.214275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:40.343924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8aee4f-6cd7-4f5f-9134-e428f474183d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:40.421412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96449d2-31a7-414b-a171-d988543c7853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:40.422233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96449d2-31a7-414b-a171-d988543c7853 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:40.422515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96449d2-31a7-414b-a171-d988543c7853 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:40.422821 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96449d2-31a7-414b-a171-d988543c7853 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:40.447935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96449d2-31a7-414b-a171-d988543c7853 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:40.456668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-d96449d2-31a7-414b-a171-d988543c7853 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:40.456668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:40.456668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:40.456668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:40.456668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:40.456668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:40.456668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:40.456668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:40.456668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:40.456668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:40.456668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:40.456668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:40.456668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:40.456668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:40.456668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:40.456668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:40.457998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:40.457998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:40.457998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:40.457998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:40.457998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:40.457998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:40.457998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:40.457998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:40.457998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:40.457998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:40.457998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:40.457998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:40.457998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:40.457998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:40.457998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:40.457998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:40.459231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:40.459231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:40.459231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:40.459231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:40.459231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:40.459231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:40.459231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:40.459231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:40.459231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:40.459231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:40.459231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:40.459231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:40.462641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 190/397] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:40.464570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206c7f5-d5bf-4f9c-9601-163da85ba227 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:40.503144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:40.584694 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206c7f5-d5bf-4f9c-9601-163da85ba227 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:40.624267 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79529656-4bed-4883-a732-578a9d37d16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:40.624927 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79529656-4bed-4883-a732-578a9d37d16e None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:40.625216 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79529656-4bed-4883-a732-578a9d37d16e None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:40.625492 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79529656-4bed-4883-a732-578a9d37d16e None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:40.626246 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79529656-4bed-4883-a732-578a9d37d16e None admin] RBAC: Adding query filter params (name=member) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:40.626587 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79529656-4bed-4883-a732-578a9d37d16e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:40.627995 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79529656-4bed-4883-a732-578a9d37d16e None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:40.642670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 208/398] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:40.644119 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96449d2-31a7-414b-a171-d988543c7853 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:40.664449 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811feb3-ec1f-4d4b-9e3f-9840ece12a9b None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:40.665066 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811feb3-ec1f-4d4b-9e3f-9840ece12a9b None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:40.665066 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811feb3-ec1f-4d4b-9e3f-9840ece12a9b None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:40.666353 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 209/399] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:25:40.673410 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a499c-fa0d-4173-9632-cdb7d5254483 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:40.673703 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a499c-fa0d-4173-9632-cdb7d5254483 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:40.673984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a499c-fa0d-4173-9632-cdb7d5254483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:40.704272 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-803a499c-fa0d-4173-9632-cdb7d5254483 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:40.705474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-803a499c-fa0d-4173-9632-cdb7d5254483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:40.756661 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:40.799271 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96449d2-31a7-414b-a171-d988543c7853 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:40.810166 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 210/400] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 23 11:25:40.812669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41493eb-7085-4999-92c2-6a0e5a9c4981 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:40.813163 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41493eb-7085-4999-92c2-6a0e5a9c4981 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:40.813760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41493eb-7085-4999-92c2-6a0e5a9c4981 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:40.815650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 211/401] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:40.821305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159680b-1eaa-41a8-acf4-629a2a987153 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:40.821819 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159680b-1eaa-41a8-acf4-629a2a987153 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:40.822372 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159680b-1eaa-41a8-acf4-629a2a987153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:40.837236 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73f4d1-dcbc-4595-9ca2-41f59100b831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:40.838347 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73f4d1-dcbc-4595-9ca2-41f59100b831 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:40.838817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73f4d1-dcbc-4595-9ca2-41f59100b831 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:40.839238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73f4d1-dcbc-4595-9ca2-41f59100b831 None admin] PATH_INFO: `/v3/groups/90d4554e0dbf49d39cb31d402accb07f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:40.840664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73f4d1-dcbc-4595-9ca2-41f59100b831 None admin] RBAC: Authorizing `identity:get_group(group_id=90d4554e0dbf49d39cb31d402accb07f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:40.840892 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c159680b-1eaa-41a8-acf4-629a2a987153 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:40.842721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c159680b-1eaa-41a8-acf4-629a2a987153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:40.842997 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73f4d1-dcbc-4595-9ca2-41f59100b831 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:40.849349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 191/402] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 23 11:25:40 2022] GET /identity/v3/groups/90d4554e0dbf49d39cb31d402accb07f => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:40.850760 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73f4d1-dcbc-4595-9ca2-41f59100b831 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:40.859969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:40.919175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 212/403] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:25:40.927453 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159680b-1eaa-41a8-acf4-629a2a987153 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:40.963489 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73f4d1-dcbc-4595-9ca2-41f59100b831 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:41.001781 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e87efc-0880-4751-9e4d-63eac327668f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:41.003111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e87efc-0880-4751-9e4d-63eac327668f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:41.003581 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e87efc-0880-4751-9e4d-63eac327668f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:41.004035 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e87efc-0880-4751-9e4d-63eac327668f None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:41.005437 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e87efc-0880-4751-9e4d-63eac327668f None admin] RBAC: Authorizing `identity:get_project(project_id=b05339ac8c27421a9a1921726dcbdc7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:41.007398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e87efc-0880-4751-9e4d-63eac327668f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:41.014666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 192/404] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:40 2022] GET /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:41.015909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e87efc-0880-4751-9e4d-63eac327668f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:41.020152 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:41.037828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159680b-1eaa-41a8-acf4-629a2a987153 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:41.073267 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9623c-d939-4901-aefc-b514e1475681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:41.074356 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9623c-d939-4901-aefc-b514e1475681 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:41.074821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9623c-d939-4901-aefc-b514e1475681 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:41.075276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9623c-d939-4901-aefc-b514e1475681 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:41.086311 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9623c-d939-4901-aefc-b514e1475681 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:41.094748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-97f9623c-d939-4901-aefc-b514e1475681 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:41.094748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:41.094748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:41.094748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:41.094748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:41.094748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:41.094748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:41.094748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:41.094748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:41.094748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:41.094748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:41.094748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:41.094748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:41.094748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:41.094748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:41.094748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:41.095639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:41.095639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:41.095639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:41.095639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:41.095639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:41.095639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:41.095639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:41.095639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:41.095639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:41.095639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:41.095639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:41.095639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:41.095639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:41.095639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:41.095639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:41.095639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:41.096430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:41.096430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:41.096430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:41.096430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:41.096430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:41.096430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:41.096430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:41.096430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:41.096430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:41.096430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:41.096430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:41.096430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:41.098930 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 213/405] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:25:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:41.099858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9623c-d939-4901-aefc-b514e1475681 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:41.113401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:41.128211 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e87efc-0880-4751-9e4d-63eac327668f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:41.166623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8eb49-3084-4b4a-b2c6-d1a914fc03ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:41.167706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8eb49-3084-4b4a-b2c6-d1a914fc03ac None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:41.168236 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8eb49-3084-4b4a-b2c6-d1a914fc03ac None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:41.168657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8eb49-3084-4b4a-b2c6-d1a914fc03ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:41.180890 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec8eb49-3084-4b4a-b2c6-d1a914fc03ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:41.189525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-4ec8eb49-3084-4b4a-b2c6-d1a914fc03ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:25:41.189525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:41.189525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:41.189525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:41.189525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:41.189525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:41.189525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:41.189525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:41.189525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:41.189525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:41.189525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:41.189525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:41.189525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:41.189525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:41.189525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:41.189525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:41.190666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:41.190666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:41.190666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:41.190666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:41.190666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:41.190666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:41.190666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:41.190666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:41.190666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:41.190666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:41.190666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:41.190666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:41.190666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:41.190666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:41.190666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:41.190666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:41.191653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:41.191653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:41.191653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:41.191653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:41.191653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:41.191653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:41.191653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:41.191653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:41.191653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:41.191653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:41.191653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:25:41.191653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:41.194327 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 193/406] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:25:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:41.195553 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8eb49-3084-4b4a-b2c6-d1a914fc03ac None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:41.208937 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9623c-d939-4901-aefc-b514e1475681 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:41.243086 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cce03d-7003-4c41-b926-4db585a99dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:41.244142 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cce03d-7003-4c41-b926-4db585a99dc5 None admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:41.244492 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cce03d-7003-4c41-b926-4db585a99dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:41.244871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cce03d-7003-4c41-b926-4db585a99dc5 None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f/groups/90d4554e0dbf49d39cb31d402accb07f/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:41.247214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cce03d-7003-4c41-b926-4db585a99dc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b05339ac8c27421a9a1921726dcbdc7f, group_id=90d4554e0dbf49d39cb31d402accb07f, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:41.248872 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cce03d-7003-4c41-b926-4db585a99dc5 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:41.268912 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:41.308801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8eb49-3084-4b4a-b2c6-d1a914fc03ac None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:41.342033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd8f01-35a8-4b5c-92b4-6603ef3ce392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:41.342718 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd8f01-35a8-4b5c-92b4-6603ef3ce392 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:41.343005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd8f01-35a8-4b5c-92b4-6603ef3ce392 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:41.343284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd8f01-35a8-4b5c-92b4-6603ef3ce392 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:41.344028 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd8f01-35a8-4b5c-92b4-6603ef3ce392 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:41.344336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd8f01-35a8-4b5c-92b4-6603ef3ce392 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:41.345725 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd8f01-35a8-4b5c-92b4-6603ef3ce392 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:41.354189 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 194/407] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:25:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:41.355161 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd8f01-35a8-4b5c-92b4-6603ef3ce392 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:41.362265 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 214/408] 10.222.0.13 () {58 vars in 1442 bytes} [Thu Jun 23 11:25:41 2022] PUT /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f/groups/90d4554e0dbf49d39cb31d402accb07f/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:25:41.367902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cce03d-7003-4c41-b926-4db585a99dc5 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:41.522900 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:41.538288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd8f01-35a8-4b5c-92b4-6603ef3ce392 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:41.556530 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cce03d-7003-4c41-b926-4db585a99dc5 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:41.567716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:41.579627 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d505c4-433e-41e6-8d77-ea06448a3fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:41.580696 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d505c4-433e-41e6-8d77-ea06448a3fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:41.580696 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d505c4-433e-41e6-8d77-ea06448a3fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:41.580890 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d505c4-433e-41e6-8d77-ea06448a3fa4 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:41.581557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d505c4-433e-41e6-8d77-ea06448a3fa4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:41.581866 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d505c4-433e-41e6-8d77-ea06448a3fa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:41.583351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d505c4-433e-41e6-8d77-ea06448a3fa4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:41.593144 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 195/409] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:41.594575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc455c31-557f-479a-b64c-9e2e3dc0092e None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:41.594853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc455c31-557f-479a-b64c-9e2e3dc0092e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:41.595135 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc455c31-557f-479a-b64c-9e2e3dc0092e None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:41.596429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 196/410] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:41.598385 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc455c31-557f-479a-b64c-9e2e3dc0092e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:41.608702 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83472078-35e6-436f-b4f4-c2043c3c189b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:41.610118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83472078-35e6-436f-b4f4-c2043c3c189b None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:41.610406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83472078-35e6-436f-b4f4-c2043c3c189b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:41.610891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83472078-35e6-436f-b4f4-c2043c3c189b None admin] PATH_INFO: `/v3/groups/90d4554e0dbf49d39cb31d402accb07f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:41.612464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83472078-35e6-436f-b4f4-c2043c3c189b None admin] RBAC: Authorizing `identity:get_group(group_id=90d4554e0dbf49d39cb31d402accb07f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:41.614691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83472078-35e6-436f-b4f4-c2043c3c189b None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:41.616816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 215/411] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 23 11:25:41 2022] GET /identity/v3/groups/90d4554e0dbf49d39cb31d402accb07f => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:41.618259 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e3ce1-2989-4c1c-a42b-1c5ad664fa14 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:41.618703 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e3ce1-2989-4c1c-a42b-1c5ad664fa14 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:41.618989 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e3ce1-2989-4c1c-a42b-1c5ad664fa14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:41.642978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-277e3ce1-2989-4c1c-a42b-1c5ad664fa14 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:41.644225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-277e3ce1-2989-4c1c-a42b-1c5ad664fa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:41.714921 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc455c31-557f-479a-b64c-9e2e3dc0092e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:41.723427 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 216/412] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:41.724283 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e3ce1-2989-4c1c-a42b-1c5ad664fa14 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:41.754084 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472075a2-cd77-496e-b509-cad666136824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:41.754982 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472075a2-cd77-496e-b509-cad666136824 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:41.755267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472075a2-cd77-496e-b509-cad666136824 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:41.755539 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472075a2-cd77-496e-b509-cad666136824 None admin] PATH_INFO: `/v3/groups/2a271c10c7e1469bbae9bdea7c6cc43f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:41.756720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472075a2-cd77-496e-b509-cad666136824 None admin] RBAC: Authorizing `identity:get_group(group_id=2a271c10c7e1469bbae9bdea7c6cc43f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:41.758204 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472075a2-cd77-496e-b509-cad666136824 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:41.759887 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 197/413] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 23 11:25:41 2022] GET /identity/v3/groups/2a271c10c7e1469bbae9bdea7c6cc43f => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:41.766050 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472075a2-cd77-496e-b509-cad666136824 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:41.803695 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:41.827238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:41.853609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e3ce1-2989-4c1c-a42b-1c5ad664fa14 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:41.874714 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472075a2-cd77-496e-b509-cad666136824 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:41.904635 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea78503-fd9f-4923-930b-900210a253b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:41.906924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea78503-fd9f-4923-930b-900210a253b6 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:41.907557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea78503-fd9f-4923-930b-900210a253b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:41.908136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea78503-fd9f-4923-930b-900210a253b6 None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:41.910206 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea78503-fd9f-4923-930b-900210a253b6 None admin] RBAC: Authorizing `identity:get_project(project_id=b05339ac8c27421a9a1921726dcbdc7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:41.912308 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea78503-fd9f-4923-930b-900210a253b6 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:41.914458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6224175-aba2-49ef-93f5-0c2ed26debea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:41.915036 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 217/414] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:41 2022] GET /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:41.915275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6224175-aba2-49ef-93f5-0c2ed26debea None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:41.915654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6224175-aba2-49ef-93f5-0c2ed26debea None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:41.915930 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6224175-aba2-49ef-93f5-0c2ed26debea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:41.916838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea78503-fd9f-4923-930b-900210a253b6 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:41.929378 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6224175-aba2-49ef-93f5-0c2ed26debea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:41.938690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-a6224175-aba2-49ef-93f5-0c2ed26debea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:41.938690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:41.938690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:41.938690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:41.938690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:41.938690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:41.938690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:41.938690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:41.938690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:41.938690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:41.938690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:41.938690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:41.938690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:41.938690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:41.938690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:41.938690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:41.939580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:41.939580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:41.939580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:41.939580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:41.939580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:41.939580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:41.939580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:41.939580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:41.939580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:41.939580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:41.939580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:41.939580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:41.939580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:41.939580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:41.939580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:41.939580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:41.940381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:41.940381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:41.940381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:41.940381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:41.940381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:41.940381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:41.940381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:41.940381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:41.940381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:41.940381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:41.940381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:41.940381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:41.945107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 198/415] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:25:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:41.945691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6224175-aba2-49ef-93f5-0c2ed26debea None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:42.053853 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea78503-fd9f-4923-930b-900210a253b6 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:42.060918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6224175-aba2-49ef-93f5-0c2ed26debea None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:42.079793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:42.099908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd0d07-df78-4357-9df2-a59b63d14c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:42.100903 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd0d07-df78-4357-9df2-a59b63d14c31 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:42.103591 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd0d07-df78-4357-9df2-a59b63d14c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:42.103591 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd0d07-df78-4357-9df2-a59b63d14c31 None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f/groups/90d4554e0dbf49d39cb31d402accb07f/roles/25e99b2a08914aa0ab18cc4955f25364` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:42.103794 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936bd2ca-cc5a-4c19-94fc-b47bf70a77eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:42.104059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd0d07-df78-4357-9df2-a59b63d14c31 None admin] RBAC: Authorizing `identity:create_grant(project_id=b05339ac8c27421a9a1921726dcbdc7f, group_id=90d4554e0dbf49d39cb31d402accb07f, role_id=25e99b2a08914aa0ab18cc4955f25364)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:42.104324 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bd2ca-cc5a-4c19-94fc-b47bf70a77eb None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:42.104740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bd2ca-cc5a-4c19-94fc-b47bf70a77eb None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:42.105164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bd2ca-cc5a-4c19-94fc-b47bf70a77eb None admin] PATH_INFO: `/v3/projects/237fd59b9b4946f0a2bdfcf73db9274e` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:42.105894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd0d07-df78-4357-9df2-a59b63d14c31 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:42.106687 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936bd2ca-cc5a-4c19-94fc-b47bf70a77eb None admin] RBAC: Authorizing `identity:get_project(project_id=237fd59b9b4946f0a2bdfcf73db9274e)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:42.108474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936bd2ca-cc5a-4c19-94fc-b47bf70a77eb None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:42.111138 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 218/416] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:41 2022] GET /identity/v3/projects/237fd59b9b4946f0a2bdfcf73db9274e => generated 344 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:42.112150 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936bd2ca-cc5a-4c19-94fc-b47bf70a77eb None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:42.116198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:42.177909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 199/417] 10.222.0.13 () {58 vars in 1442 bytes} [Thu Jun 23 11:25:41 2022] PUT /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f/groups/90d4554e0dbf49d39cb31d402accb07f/roles/25e99b2a08914aa0ab18cc4955f25364 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:25:42.179097 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd0d07-df78-4357-9df2-a59b63d14c31 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:42.285592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936bd2ca-cc5a-4c19-94fc-b47bf70a77eb None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:42.350959 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de93e7e-378e-4f36-85fc-43ef012a3524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:42.351631 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de93e7e-378e-4f36-85fc-43ef012a3524 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:42.351916 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de93e7e-378e-4f36-85fc-43ef012a3524 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:42.352193 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de93e7e-378e-4f36-85fc-43ef012a3524 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:42.352938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de93e7e-378e-4f36-85fc-43ef012a3524 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:42.353304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de93e7e-378e-4f36-85fc-43ef012a3524 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:42.363005 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de93e7e-378e-4f36-85fc-43ef012a3524 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:42.369575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd0d07-df78-4357-9df2-a59b63d14c31 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:42.375377 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 219/418] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:25:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:42.383963 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de93e7e-378e-4f36-85fc-43ef012a3524 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:42.417194 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf525a-9316-401f-af97-ad57f586f495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:42.418190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf525a-9316-401f-af97-ad57f586f495 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:42.418477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf525a-9316-401f-af97-ad57f586f495 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:42.418793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf525a-9316-401f-af97-ad57f586f495 None admin] PATH_INFO: `/v3/projects/237fd59b9b4946f0a2bdfcf73db9274e/groups/2a271c10c7e1469bbae9bdea7c6cc43f/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:42.421204 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf525a-9316-401f-af97-ad57f586f495 None admin] RBAC: Authorizing `identity:create_grant(project_id=237fd59b9b4946f0a2bdfcf73db9274e, group_id=2a271c10c7e1469bbae9bdea7c6cc43f, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:42.422952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf525a-9316-401f-af97-ad57f586f495 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:42.464069 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:42.515379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de93e7e-378e-4f36-85fc-43ef012a3524 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:42.517956 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 200/419] 10.222.0.13 () {58 vars in 1442 bytes} [Thu Jun 23 11:25:42 2022] PUT /identity/v3/projects/237fd59b9b4946f0a2bdfcf73db9274e/groups/2a271c10c7e1469bbae9bdea7c6cc43f/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:25:42.564672 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2d8c3-8c60-4b3a-9636-16b86251269d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:42.565357 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2d8c3-8c60-4b3a-9636-16b86251269d None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:42.565639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2d8c3-8c60-4b3a-9636-16b86251269d None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:42.565968 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2d8c3-8c60-4b3a-9636-16b86251269d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:42.653839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2d8c3-8c60-4b3a-9636-16b86251269d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:42.655900 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2d8c3-8c60-4b3a-9636-16b86251269d None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:42.752795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:42.770718 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47338c0a-23af-4162-8e14-4e1779da420f None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:42.771224 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47338c0a-23af-4162-8e14-4e1779da420f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:42.771298 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47338c0a-23af-4162-8e14-4e1779da420f None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:42.772801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 201/420] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:25:42.780225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589e467-70ea-477c-985a-1cb309c5e7c4 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:42.780892 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589e467-70ea-477c-985a-1cb309c5e7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:42.781643 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589e467-70ea-477c-985a-1cb309c5e7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:42.804933 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 220/421] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 11:25:42.805583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5589e467-70ea-477c-985a-1cb309c5e7c4 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:42.807069 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5589e467-70ea-477c-985a-1cb309c5e7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:42.812381 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2d8c3-8c60-4b3a-9636-16b86251269d None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:42.843977 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:42.894886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 202/422] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:25:42.921314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589e467-70ea-477c-985a-1cb309c5e7c4 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:42.960306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2d8c3-8c60-4b3a-9636-16b86251269d None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:42.998490 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f633fae-dbd5-4782-bc12-623768205f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:42.999482 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f633fae-dbd5-4782-bc12-623768205f92 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:42.999764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f633fae-dbd5-4782-bc12-623768205f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:43.000039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f633fae-dbd5-4782-bc12-623768205f92 None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:43.001368 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f633fae-dbd5-4782-bc12-623768205f92 None admin] RBAC: Authorizing `identity:get_project(project_id=d20d807ab0054a0d994c8b287d262cba)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:43.002867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f633fae-dbd5-4782-bc12-623768205f92 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:43.004917 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 221/423] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:42 2022] GET /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:43.009951 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f633fae-dbd5-4782-bc12-623768205f92 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:43.052666 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:43.124828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f633fae-dbd5-4782-bc12-623768205f92 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:43.150307 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589e467-70ea-477c-985a-1cb309c5e7c4 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:43.162800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14cfaa-7f60-4928-a5c2-624a72e15945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:43.164017 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14cfaa-7f60-4928-a5c2-624a72e15945 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:43.164501 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14cfaa-7f60-4928-a5c2-624a72e15945 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:43.164973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14cfaa-7f60-4928-a5c2-624a72e15945 None admin] PATH_INFO: `/v3/groups/90d4554e0dbf49d39cb31d402accb07f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:43.166410 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14cfaa-7f60-4928-a5c2-624a72e15945 None admin] RBAC: Authorizing `identity:get_group(group_id=90d4554e0dbf49d39cb31d402accb07f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:43.168101 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14cfaa-7f60-4928-a5c2-624a72e15945 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:43.171219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 222/424] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 23 11:25:43 2022] GET /identity/v3/groups/90d4554e0dbf49d39cb31d402accb07f => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:43.183379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14cfaa-7f60-4928-a5c2-624a72e15945 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:43.198174 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2703f-48b2-4893-be20-0da85089f674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:43.199033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2703f-48b2-4893-be20-0da85089f674 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:43.199316 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2703f-48b2-4893-be20-0da85089f674 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:43.199591 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2703f-48b2-4893-be20-0da85089f674 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:43.211483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2703f-48b2-4893-be20-0da85089f674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:43.222571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-a1e2703f-48b2-4893-be20-0da85089f674 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:43.222571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:43.222571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:43.222571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:43.222571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:43.222571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:43.222571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:43.222571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:43.222571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:43.222571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:43.222571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:43.222571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:43.222571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:43.222571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:43.222571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:43.222571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:43.223526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:43.223526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:43.223526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:43.223526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:43.223526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:43.223526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:43.223526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:43.223526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:43.223526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:43.223526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:43.223526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:43.223526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:43.223526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:43.223526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:43.223526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:43.223526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:43.224314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:43.224314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:43.224314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:43.224314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:43.224314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:43.224314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:43.224314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:43.224314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:43.224314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:43.224314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:43.224314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:43.224314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:43.224314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 203/425] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:25:43.230565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2703f-48b2-4893-be20-0da85089f674 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:43.256833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:43.355716 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14cfaa-7f60-4928-a5c2-624a72e15945 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:43.358852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:43.364448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2703f-48b2-4893-be20-0da85089f674 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:43.394200 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d938b09-4801-4aee-bb85-d900f16c2c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:43.395349 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d938b09-4801-4aee-bb85-d900f16c2c93 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:43.395641 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d938b09-4801-4aee-bb85-d900f16c2c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:43.395919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d938b09-4801-4aee-bb85-d900f16c2c93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:43.396749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d938b09-4801-4aee-bb85-d900f16c2c93 None admin] RBAC: Adding query filter params (group.id=90d4554e0dbf49d39cb31d402accb07f, role.id=25e99b2a08914aa0ab18cc4955f25364, scope.project.id=d20d807ab0054a0d994c8b287d262cba) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:43.397068 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d938b09-4801-4aee-bb85-d900f16c2c93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:43.398506 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d938b09-4801-4aee-bb85-d900f16c2c93 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:43.405828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 223/426] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jun 23 11:25:43 2022] GET /identity/v3/role_assignments?group.id=90d4554e0dbf49d39cb31d402accb07f&scope.project.id=d20d807ab0054a0d994c8b287d262cba&role.id=25e99b2a08914aa0ab18cc4955f25364 => generated 611 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:43.459573 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927e35c-14c9-46d4-a10b-e0f13f2e1773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:43.460190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927e35c-14c9-46d4-a10b-e0f13f2e1773 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:43.460467 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927e35c-14c9-46d4-a10b-e0f13f2e1773 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:43.460739 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927e35c-14c9-46d4-a10b-e0f13f2e1773 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:43.461502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e927e35c-14c9-46d4-a10b-e0f13f2e1773 None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:43.461803 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e927e35c-14c9-46d4-a10b-e0f13f2e1773 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:43.463210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e927e35c-14c9-46d4-a10b-e0f13f2e1773 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:43.474028 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 204/427] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:43.479248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d938b09-4801-4aee-bb85-d900f16c2c93 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:43.618334 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d938b09-4801-4aee-bb85-d900f16c2c93 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:43.656054 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5f9e1-7307-451f-b812-7dcea0caeb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:43.656758 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5f9e1-7307-451f-b812-7dcea0caeb01 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:43.657044 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5f9e1-7307-451f-b812-7dcea0caeb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:43.657320 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5f9e1-7307-451f-b812-7dcea0caeb01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:43.693816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:43.713697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437056ae-14d8-4f85-b851-de97ceccfb31 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:43.713987 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437056ae-14d8-4f85-b851-de97ceccfb31 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:43.714179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437056ae-14d8-4f85-b851-de97ceccfb31 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:43.715819 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 205/428] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:43.721803 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e7460-ac42-4bd7-850a-93a7938168ff None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:43.722095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e7460-ac42-4bd7-850a-93a7938168ff None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:43.722383 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e7460-ac42-4bd7-850a-93a7938168ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:43.736622 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5f9e1-7307-451f-b812-7dcea0caeb01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:43.738396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5f9e1-7307-451f-b812-7dcea0caeb01 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:43.741822 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ce0e7460-ac42-4bd7-850a-93a7938168ff None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:43.742577 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ce0e7460-ac42-4bd7-850a-93a7938168ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:43.819132 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 206/429] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:43.843778 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e7460-ac42-4bd7-850a-93a7938168ff None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:43.848460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 224/430] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:43.855566 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5f9e1-7307-451f-b812-7dcea0caeb01 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:43.866415 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:43.933800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:43.962668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e7460-ac42-4bd7-850a-93a7938168ff None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:43.967517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5f9e1-7307-451f-b812-7dcea0caeb01 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:44.001611 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed4616-38d9-4123-a932-3e6de68fe0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:44.002408 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed4616-38d9-4123-a932-3e6de68fe0fb None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:44.002715 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed4616-38d9-4123-a932-3e6de68fe0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:44.002984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed4616-38d9-4123-a932-3e6de68fe0fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:44.006457 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c3282-2c58-4f2d-b071-2395db199997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:44.007498 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c3282-2c58-4f2d-b071-2395db199997 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:44.007924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c3282-2c58-4f2d-b071-2395db199997 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:44.008341 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c3282-2c58-4f2d-b071-2395db199997 None admin] PATH_INFO: `/v3/projects/d20d807ab0054a0d994c8b287d262cba` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:44.009758 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c3282-2c58-4f2d-b071-2395db199997 None admin] RBAC: Authorizing `identity:get_project(project_id=d20d807ab0054a0d994c8b287d262cba)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:44.011600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c3282-2c58-4f2d-b071-2395db199997 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:44.013713 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed4616-38d9-4123-a932-3e6de68fe0fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:44.013931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 225/431] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:43 2022] GET /identity/v3/projects/d20d807ab0054a0d994c8b287d262cba => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:44.021500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-44ed4616-38d9-4123-a932-3e6de68fe0fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:44.021500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:44.021500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:44.021500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:44.021500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:44.021500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:44.021500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:44.021500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:44.021500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:44.021500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:44.021500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:44.021500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:44.021500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:44.021500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:44.021500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:44.021500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:44.022637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:44.022637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:44.022637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:44.022637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:44.022637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:44.022637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:44.022637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:44.022637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:44.022637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:44.022637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:44.022637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:44.022637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:44.022637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:44.022637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:44.022637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:44.022637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:44.023617 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:44.023617 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:44.023617 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:44.023617 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:44.023617 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:44.023617 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:44.023617 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:44.023617 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:44.023617 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:44.023617 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:44.023617 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:25:44.023617 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:44.023617 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 207/432] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:44.025316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c3282-2c58-4f2d-b071-2395db199997 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:44.035795 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed4616-38d9-4123-a932-3e6de68fe0fb None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:44.141364 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c3282-2c58-4f2d-b071-2395db199997 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:44.147686 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:44.177913 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0a157-b5b8-465c-8abe-f5868d722190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:44.178405 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed4616-38d9-4123-a932-3e6de68fe0fb None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:44.178812 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0a157-b5b8-465c-8abe-f5868d722190 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:44.179096 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0a157-b5b8-465c-8abe-f5868d722190 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:44.179373 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0a157-b5b8-465c-8abe-f5868d722190 None admin] PATH_INFO: `/v3/groups/90d4554e0dbf49d39cb31d402accb07f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:44.180728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0a157-b5b8-465c-8abe-f5868d722190 None admin] RBAC: Authorizing `identity:get_group(group_id=90d4554e0dbf49d39cb31d402accb07f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:44.182990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0a157-b5b8-465c-8abe-f5868d722190 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:44.184997 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 226/433] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 23 11:25:44 2022] GET /identity/v3/groups/90d4554e0dbf49d39cb31d402accb07f => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:44.191036 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0a157-b5b8-465c-8abe-f5868d722190 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:44.194174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:44.227399 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74e092-73de-4c45-ae3a-a0ef0eab376f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:44.230095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74e092-73de-4c45-ae3a-a0ef0eab376f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:44.230518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74e092-73de-4c45-ae3a-a0ef0eab376f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:44.230994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74e092-73de-4c45-ae3a-a0ef0eab376f None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:44.231894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74e092-73de-4c45-ae3a-a0ef0eab376f None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:44.232340 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74e092-73de-4c45-ae3a-a0ef0eab376f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:44.234204 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74e092-73de-4c45-ae3a-a0ef0eab376f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:44.247585 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 208/434] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:25:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:44.254736 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74e092-73de-4c45-ae3a-a0ef0eab376f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:44.302693 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0a157-b5b8-465c-8abe-f5868d722190 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:44.339204 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19cadf3-331b-4129-aa3a-2a4c054e058e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:44.339858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19cadf3-331b-4129-aa3a-2a4c054e058e None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:44.340139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19cadf3-331b-4129-aa3a-2a4c054e058e None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:44.340412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19cadf3-331b-4129-aa3a-2a4c054e058e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:44.341242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19cadf3-331b-4129-aa3a-2a4c054e058e None admin] RBAC: Adding query filter params (group.id=90d4554e0dbf49d39cb31d402accb07f, role.id=75f08cd0671f436c9f968976aa8baa7f, scope.project.id=d20d807ab0054a0d994c8b287d262cba) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:44.341554 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19cadf3-331b-4129-aa3a-2a4c054e058e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:44.344546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19cadf3-331b-4129-aa3a-2a4c054e058e None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:44.356928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 227/435] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jun 23 11:25:44 2022] GET /identity/v3/role_assignments?group.id=90d4554e0dbf49d39cb31d402accb07f&scope.project.id=d20d807ab0054a0d994c8b287d262cba&role.id=75f08cd0671f436c9f968976aa8baa7f => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:44.374683 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74e092-73de-4c45-ae3a-a0ef0eab376f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:44.414377 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08fd378-12f5-4b9f-a4c9-85df459fee9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:44.415469 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fd378-12f5-4b9f-a4c9-85df459fee9b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:44.415947 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fd378-12f5-4b9f-a4c9-85df459fee9b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:44.416411 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fd378-12f5-4b9f-a4c9-85df459fee9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:44.429093 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:44.498774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08fd378-12f5-4b9f-a4c9-85df459fee9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:44.498774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08fd378-12f5-4b9f-a4c9-85df459fee9b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:44.604798 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 209/436] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 11:25:44.611541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19cadf3-331b-4129-aa3a-2a4c054e058e None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:44.665405 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f33762-abe6-4fbb-b9f4-78a5e2ec7a52 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:44.665719 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f33762-abe6-4fbb-b9f4-78a5e2ec7a52 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:44.665957 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f33762-abe6-4fbb-b9f4-78a5e2ec7a52 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:44.667350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 210/437] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:25:44.674792 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b46ae-ccee-46cc-8aae-241db4a5f94e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:44.675076 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b46ae-ccee-46cc-8aae-241db4a5f94e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:44.675346 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b46ae-ccee-46cc-8aae-241db4a5f94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:44.688464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:44.702071 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-296b46ae-ccee-46cc-8aae-241db4a5f94e None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:44.703292 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-296b46ae-ccee-46cc-8aae-241db4a5f94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:44.736537 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19cadf3-331b-4129-aa3a-2a4c054e058e None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:44.754377 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:44.781274 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db91da-d23e-4681-8d17-a499e7322249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:44.782151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db91da-d23e-4681-8d17-a499e7322249 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:44.782435 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db91da-d23e-4681-8d17-a499e7322249 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:44.782742 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db91da-d23e-4681-8d17-a499e7322249 None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:44.784410 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db91da-d23e-4681-8d17-a499e7322249 None admin] RBAC: Authorizing `identity:get_project(project_id=b05339ac8c27421a9a1921726dcbdc7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:44.785867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db91da-d23e-4681-8d17-a499e7322249 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:44.788022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 228/438] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:44 2022] GET /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:44.792964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db91da-d23e-4681-8d17-a499e7322249 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:44.799612 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 211/439] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:25:44.824031 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b46ae-ccee-46cc-8aae-241db4a5f94e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:44.920690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db91da-d23e-4681-8d17-a499e7322249 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:44.963065 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b46ae-ccee-46cc-8aae-241db4a5f94e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:44.966654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bfad8-2b49-4b02-a26b-e69921dda6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:44.969510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bfad8-2b49-4b02-a26b-e69921dda6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:44.969510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bfad8-2b49-4b02-a26b-e69921dda6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:44.969510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bfad8-2b49-4b02-a26b-e69921dda6a9 None admin] PATH_INFO: `/v3/groups/90d4554e0dbf49d39cb31d402accb07f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:44.971286 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641bfad8-2b49-4b02-a26b-e69921dda6a9 None admin] RBAC: Authorizing `identity:get_group(group_id=90d4554e0dbf49d39cb31d402accb07f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:44.973490 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641bfad8-2b49-4b02-a26b-e69921dda6a9 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:44.975618 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 229/440] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 23 11:25:44 2022] GET /identity/v3/groups/90d4554e0dbf49d39cb31d402accb07f => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:44.977555 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251fedf-bdef-4504-8b59-49201c063bbe None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:44.977850 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251fedf-bdef-4504-8b59-49201c063bbe None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:44.978124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251fedf-bdef-4504-8b59-49201c063bbe None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:44.979575 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 230/441] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:44.981898 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251fedf-bdef-4504-8b59-49201c063bbe None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:44.985143 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:45.016510 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7db7cf-d514-4b6f-ba90-8eeaef26bf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:45.016510 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7db7cf-d514-4b6f-ba90-8eeaef26bf24 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:45.016510 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7db7cf-d514-4b6f-ba90-8eeaef26bf24 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:45.016510 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7db7cf-d514-4b6f-ba90-8eeaef26bf24 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:45.020881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7db7cf-d514-4b6f-ba90-8eeaef26bf24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:45.029079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-ce7db7cf-d514-4b6f-ba90-8eeaef26bf24 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:45.029079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:45.029079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:45.029079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:45.029079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:45.029079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:45.029079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:45.029079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:45.029079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:45.029079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:45.029079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:45.029079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:45.029079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:45.029079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:45.029079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:45.029079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:45.030717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:45.030717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:45.030717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:45.030717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:45.030717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:45.030717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:45.030717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:45.030717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:45.030717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:45.030717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:45.030717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:45.030717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:45.030717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:45.030717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:45.030717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:45.030717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:45.031480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:45.031480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:45.031480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:45.031480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:45.031480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:45.031480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:45.031480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:45.031480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:45.031480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:45.031480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:45.031480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:45.031480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:45.031480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 212/442] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:25:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:45.033111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c37928-4835-49fa-8c14-5ab0540bb332 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:45.033111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c37928-4835-49fa-8c14-5ab0540bb332 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:45.033111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c37928-4835-49fa-8c14-5ab0540bb332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:45.053061 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f7c37928-4835-49fa-8c14-5ab0540bb332 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:45.054240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f7c37928-4835-49fa-8c14-5ab0540bb332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:45.079935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:45.100399 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251fedf-bdef-4504-8b59-49201c063bbe None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:45.122006 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 213/443] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:45.123034 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c37928-4835-49fa-8c14-5ab0540bb332 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:45.135333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9f6a5-3ffd-41ec-bb26-c94c2b4f7794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:45.136122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9f6a5-3ffd-41ec-bb26-c94c2b4f7794 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:45.136489 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9f6a5-3ffd-41ec-bb26-c94c2b4f7794 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:45.136795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9f6a5-3ffd-41ec-bb26-c94c2b4f7794 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:45.137631 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9f6a5-3ffd-41ec-bb26-c94c2b4f7794 None admin] RBAC: Adding query filter params (group.id=90d4554e0dbf49d39cb31d402accb07f, role.id=75f08cd0671f436c9f968976aa8baa7f, scope.project.id=b05339ac8c27421a9a1921726dcbdc7f) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:45.137969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9f6a5-3ffd-41ec-bb26-c94c2b4f7794 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:45.140125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9f6a5-3ffd-41ec-bb26-c94c2b4f7794 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:45.146586 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 231/444] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jun 23 11:25:44 2022] GET /identity/v3/role_assignments?group.id=90d4554e0dbf49d39cb31d402accb07f&scope.project.id=b05339ac8c27421a9a1921726dcbdc7f&role.id=75f08cd0671f436c9f968976aa8baa7f => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:45.147412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9f6a5-3ffd-41ec-bb26-c94c2b4f7794 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:45.232802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:45.241394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c37928-4835-49fa-8c14-5ab0540bb332 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:45.278017 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9f6a5-3ffd-41ec-bb26-c94c2b4f7794 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:45.280410 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe2089-045e-4d54-84c1-1e0ad276c9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:45.281101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe2089-045e-4d54-84c1-1e0ad276c9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:45.281387 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe2089-045e-4d54-84c1-1e0ad276c9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:45.281660 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe2089-045e-4d54-84c1-1e0ad276c9d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:45.282431 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe2089-045e-4d54-84c1-1e0ad276c9d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:45.282770 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe2089-045e-4d54-84c1-1e0ad276c9d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:45.284189 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe2089-045e-4d54-84c1-1e0ad276c9d5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:45.297675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 214/445] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:25:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:45.299134 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe2089-045e-4d54-84c1-1e0ad276c9d5 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:45.328830 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6bbfc-5d4b-47e9-90c0-cb8137dbaedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:45.329630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6bbfc-5d4b-47e9-90c0-cb8137dbaedb None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:45.329920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6bbfc-5d4b-47e9-90c0-cb8137dbaedb None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:45.330202 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6bbfc-5d4b-47e9-90c0-cb8137dbaedb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:45.330352 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:45.341805 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b6bbfc-5d4b-47e9-90c0-cb8137dbaedb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:45.350149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-a4b6bbfc-5d4b-47e9-90c0-cb8137dbaedb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:25:45.350149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:45.350149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:45.350149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:45.350149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:45.350149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:45.350149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:45.350149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:45.350149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:45.350149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:45.350149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:45.350149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:45.350149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:45.350149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:45.350149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:45.350149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:45.351187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:45.351187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:45.351187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:45.351187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:45.351187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:45.351187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:45.351187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:45.351187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:45.351187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:45.351187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:45.351187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:45.351187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:45.351187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:45.351187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:45.351187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:45.351187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:45.352973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:45.352973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:45.352973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:45.352973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:45.352973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:45.352973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:45.352973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:45.352973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:45.352973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:45.352973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:45.352973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:25:45.352973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:45.352973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 232/446] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:25:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:45.357102 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6bbfc-5d4b-47e9-90c0-cb8137dbaedb None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:45.427856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe2089-045e-4d54-84c1-1e0ad276c9d5 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:45.467069 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4483d6c-b18f-408f-99c1-2b3e23eabcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:45.467994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4483d6c-b18f-408f-99c1-2b3e23eabcee None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:45.468285 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4483d6c-b18f-408f-99c1-2b3e23eabcee None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:45.468567 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4483d6c-b18f-408f-99c1-2b3e23eabcee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:45.487396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6bbfc-5d4b-47e9-90c0-cb8137dbaedb None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:45.517786 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:45.528222 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a35a9d-2d8b-4a99-8002-f18bd63fb3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:45.528874 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a35a9d-2d8b-4a99-8002-f18bd63fb3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:45.529158 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a35a9d-2d8b-4a99-8002-f18bd63fb3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:45.529474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a35a9d-2d8b-4a99-8002-f18bd63fb3d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:45.530222 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a35a9d-2d8b-4a99-8002-f18bd63fb3d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:45.530559 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a35a9d-2d8b-4a99-8002-f18bd63fb3d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:45.531999 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a35a9d-2d8b-4a99-8002-f18bd63fb3d8 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:45.540584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 233/447] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:45.546575 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a35a9d-2d8b-4a99-8002-f18bd63fb3d8 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:45.547236 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4483d6c-b18f-408f-99c1-2b3e23eabcee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:45.548721 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4483d6c-b18f-408f-99c1-2b3e23eabcee None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:45.572320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:45.645286 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a35a9d-2d8b-4a99-8002-f18bd63fb3d8 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:45.646473 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 215/448] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:45.651951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4483d6c-b18f-408f-99c1-2b3e23eabcee None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:45.679201 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396485e-1758-440c-9963-db1c95e641af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:45.679849 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396485e-1758-440c-9963-db1c95e641af None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:45.680134 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396485e-1758-440c-9963-db1c95e641af None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:45.680415 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396485e-1758-440c-9963-db1c95e641af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:45.735385 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:45.746999 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8396485e-1758-440c-9963-db1c95e641af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:45.749081 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8396485e-1758-440c-9963-db1c95e641af None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:45.749194 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4483d6c-b18f-408f-99c1-2b3e23eabcee None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:45.771426 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:45.782726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e6cc3-0456-4ea7-a081-3fd42fea9bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:45.783515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e6cc3-0456-4ea7-a081-3fd42fea9bfc None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:45.783796 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e6cc3-0456-4ea7-a081-3fd42fea9bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:45.784066 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e6cc3-0456-4ea7-a081-3fd42fea9bfc None admin] PATH_INFO: `/v3/projects/b05339ac8c27421a9a1921726dcbdc7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:45.785235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e6cc3-0456-4ea7-a081-3fd42fea9bfc None admin] RBAC: Authorizing `identity:get_project(project_id=b05339ac8c27421a9a1921726dcbdc7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:45.786695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e6cc3-0456-4ea7-a081-3fd42fea9bfc None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:45.788364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 216/449] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:45 2022] GET /identity/v3/projects/b05339ac8c27421a9a1921726dcbdc7f => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:45.793623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e6cc3-0456-4ea7-a081-3fd42fea9bfc None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:45.848071 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 234/450] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 11:25:45.853621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396485e-1758-440c-9963-db1c95e641af None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:45.892459 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e6cc3-0456-4ea7-a081-3fd42fea9bfc None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:45.926870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851000c9-4792-413b-8755-e869022380a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:45.927713 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851000c9-4792-413b-8755-e869022380a9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:45.927996 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851000c9-4792-413b-8755-e869022380a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:45.928287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851000c9-4792-413b-8755-e869022380a9 None admin] PATH_INFO: `/v3/groups/90d4554e0dbf49d39cb31d402accb07f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:45.929438 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851000c9-4792-413b-8755-e869022380a9 None admin] RBAC: Authorizing `identity:get_group(group_id=90d4554e0dbf49d39cb31d402accb07f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:45.930911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851000c9-4792-413b-8755-e869022380a9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:45.932521 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 217/451] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 23 11:25:45 2022] GET /identity/v3/groups/90d4554e0dbf49d39cb31d402accb07f => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:45.938232 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:45.938902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851000c9-4792-413b-8755-e869022380a9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:45.949967 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396485e-1758-440c-9963-db1c95e641af None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:45.983955 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f5917-640f-47d8-bcf5-0b70a7e46b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:45.984806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f5917-640f-47d8-bcf5-0b70a7e46b43 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:45.985089 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f5917-640f-47d8-bcf5-0b70a7e46b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:45.985365 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f5917-640f-47d8-bcf5-0b70a7e46b43 None admin] PATH_INFO: `/v3/projects/237fd59b9b4946f0a2bdfcf73db9274e` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:45.986548 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f5917-640f-47d8-bcf5-0b70a7e46b43 None admin] RBAC: Authorizing `identity:get_project(project_id=237fd59b9b4946f0a2bdfcf73db9274e)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:45.988002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f5917-640f-47d8-bcf5-0b70a7e46b43 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:45.989667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 235/452] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:25:45 2022] GET /identity/v3/projects/237fd59b9b4946f0a2bdfcf73db9274e => generated 344 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:45.994961 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f5917-640f-47d8-bcf5-0b70a7e46b43 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:45.997219 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:46.038614 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851000c9-4792-413b-8755-e869022380a9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:46.072244 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e6c3e-7d05-4297-8a85-39c5bf765d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:46.072904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e6c3e-7d05-4297-8a85-39c5bf765d48 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:46.073188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e6c3e-7d05-4297-8a85-39c5bf765d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:46.073470 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e6c3e-7d05-4297-8a85-39c5bf765d48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:46.074302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e6c3e-7d05-4297-8a85-39c5bf765d48 None admin] RBAC: Adding query filter params (group.id=90d4554e0dbf49d39cb31d402accb07f, role.id=25e99b2a08914aa0ab18cc4955f25364, scope.project.id=b05339ac8c27421a9a1921726dcbdc7f) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:46.074675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e6c3e-7d05-4297-8a85-39c5bf765d48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:46.076124 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e6c3e-7d05-4297-8a85-39c5bf765d48 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:46.082631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 218/453] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jun 23 11:25:45 2022] GET /identity/v3/role_assignments?group.id=90d4554e0dbf49d39cb31d402accb07f&scope.project.id=b05339ac8c27421a9a1921726dcbdc7f&role.id=25e99b2a08914aa0ab18cc4955f25364 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:46.093909 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f5917-640f-47d8-bcf5-0b70a7e46b43 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:46.131265 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180b21c-0dff-4ea9-92c2-806905260362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:46.132140 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180b21c-0dff-4ea9-92c2-806905260362 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:46.132425 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180b21c-0dff-4ea9-92c2-806905260362 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:46.132703 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180b21c-0dff-4ea9-92c2-806905260362 None admin] PATH_INFO: `/v3/groups/2a271c10c7e1469bbae9bdea7c6cc43f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:46.133984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180b21c-0dff-4ea9-92c2-806905260362 None admin] RBAC: Authorizing `identity:get_group(group_id=2a271c10c7e1469bbae9bdea7c6cc43f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:46.135524 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180b21c-0dff-4ea9-92c2-806905260362 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:46.137448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 236/454] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 23 11:25:45 2022] GET /identity/v3/groups/2a271c10c7e1469bbae9bdea7c6cc43f => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:46.143878 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e6c3e-7d05-4297-8a85-39c5bf765d48 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:46.249228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e6c3e-7d05-4297-8a85-39c5bf765d48 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:46.273545 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:46.285244 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb5af5-92d7-4d11-ab75-7264970cdefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:46.286923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb5af5-92d7-4d11-ab75-7264970cdefc None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:46.286923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb5af5-92d7-4d11-ab75-7264970cdefc None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:46.286923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb5af5-92d7-4d11-ab75-7264970cdefc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:46.287333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb5af5-92d7-4d11-ab75-7264970cdefc None admin] RBAC: Adding query filter params (group.id=2a271c10c7e1469bbae9bdea7c6cc43f, role.id=c9e006de0d6c4812ab29d778e9c8a830, scope.project.id=237fd59b9b4946f0a2bdfcf73db9274e) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:46.287648 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb5af5-92d7-4d11-ab75-7264970cdefc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:46.289148 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb5af5-92d7-4d11-ab75-7264970cdefc None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:46.295522 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 219/455] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jun 23 11:25:46 2022] GET /identity/v3/role_assignments?group.id=2a271c10c7e1469bbae9bdea7c6cc43f&scope.project.id=237fd59b9b4946f0a2bdfcf73db9274e&role.id=c9e006de0d6c4812ab29d778e9c8a830 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:48.989531 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3406e-b85a-4c34-b2a8-1c139ba14a79 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:48.990523 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3406e-b85a-4c34-b2a8-1c139ba14a79 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:48.991052 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3406e-b85a-4c34-b2a8-1c139ba14a79 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:48.992881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 237/456] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:48.995424 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92742651-2e15-4b8a-81cc-b8e1513a247d None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:48.996251 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92742651-2e15-4b8a-81cc-b8e1513a247d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:48.996748 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92742651-2e15-4b8a-81cc-b8e1513a247d None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:48.999958 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91623bf4-6bba-4ad8-b887-bc9adecfafe8 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:49.000404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 220/457] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:25:49.000907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91623bf4-6bba-4ad8-b887-bc9adecfafe8 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:49.001405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91623bf4-6bba-4ad8-b887-bc9adecfafe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:49.004092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb145db-b824-4e2c-a371-44611b670754 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:49.004589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb145db-b824-4e2c-a371-44611b670754 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:49.005064 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb145db-b824-4e2c-a371-44611b670754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:49.035518 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-91623bf4-6bba-4ad8-b887-bc9adecfafe8 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:49.036771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-9bb145db-b824-4e2c-a371-44611b670754 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:49.038208 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9bb145db-b824-4e2c-a371-44611b670754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:49.039593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-91623bf4-6bba-4ad8-b887-bc9adecfafe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:49.062114 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:49.116100 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 238/458] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:49.117800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9c466-e777-4aeb-a763-348b0457b945 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:49.118297 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9c466-e777-4aeb-a763-348b0457b945 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:49.118799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9c466-e777-4aeb-a763-348b0457b945 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:49.120648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 239/459] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:49.130554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 221/460] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:49.136012 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef815c3-cebb-4ea4-b1a1-713766c6c1f3 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:49.136533 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef815c3-cebb-4ea4-b1a1-713766c6c1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:49.139894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef815c3-cebb-4ea4-b1a1-713766c6c1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:49.143849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb145db-b824-4e2c-a371-44611b670754 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:49.159985 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-6ef815c3-cebb-4ea4-b1a1-713766c6c1f3 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:49.161968 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-6ef815c3-cebb-4ea4-b1a1-713766c6c1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:49.254711 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 240/461] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:49.255553 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef815c3-cebb-4ea4-b1a1-713766c6c1f3 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:49.294867 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb145db-b824-4e2c-a371-44611b670754 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:49.313610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:49.333876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed15f21-ca9d-448c-969c-ba9b32c95697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:49.335061 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed15f21-ca9d-448c-969c-ba9b32c95697 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:49.335589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed15f21-ca9d-448c-969c-ba9b32c95697 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:49.336157 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed15f21-ca9d-448c-969c-ba9b32c95697 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:49.347702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed15f21-ca9d-448c-969c-ba9b32c95697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:49.349783 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed15f21-ca9d-448c-969c-ba9b32c95697 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:49.363443 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:49.371372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-aed15f21-ca9d-448c-969c-ba9b32c95697 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:49.371372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:49.371372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:49.371372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:49.371372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:49.371372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:49.371372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:49.371372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:49.371372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:49.371372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:49.371372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:49.371372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:49.371372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:25:49.371372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:25:49.371372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:25:49.375190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:25:49.375190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:49.375190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:49.375190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:49.375190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:49.375190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:49.375190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:49.375190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:49.375190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:49.375190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:49.375190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:49.375190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:49.375190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:49.375190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:49.375190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:49.375190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:49.375190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:49.376036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:25:49.376036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:25:49.376036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:49.376036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:49.376036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 222/462] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:25:49.376036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed15f21-ca9d-448c-969c-ba9b32c95697 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:49.379350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef815c3-cebb-4ea4-b1a1-713766c6c1f3 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:49.424693 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63833a88-7b9c-4e0b-a0f8-32e4a34edc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:49.425489 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63833a88-7b9c-4e0b-a0f8-32e4a34edc49 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:49.425775 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63833a88-7b9c-4e0b-a0f8-32e4a34edc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:49.426051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63833a88-7b9c-4e0b-a0f8-32e4a34edc49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:49.437498 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63833a88-7b9c-4e0b-a0f8-32e4a34edc49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:49.438982 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63833a88-7b9c-4e0b-a0f8-32e4a34edc49 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:49.457487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-63833a88-7b9c-4e0b-a0f8-32e4a34edc49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:49.457487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:49.457487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:49.457487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:49.457487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:49.457487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:49.457487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:49.457487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:49.457487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:49.457487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:49.457487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:49.457487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:49.457487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:25:49.457487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:25:49.457487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:25:49.458417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:25:49.458417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:49.458417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:49.458417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:49.458417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:49.458417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:49.458417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:49.458417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:49.458417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:49.458417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:49.458417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:49.458417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:49.458417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:49.458417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:49.458417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:49.458417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:49.458417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:49.459350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:25:49.459350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:25:49.459350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:49.459350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:49.459350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 241/463] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:49.459604 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63833a88-7b9c-4e0b-a0f8-32e4a34edc49 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:49.502833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed15f21-ca9d-448c-969c-ba9b32c95697 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:49.540512 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0514c-d0ad-434d-92bf-1a65d340d536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:49.541243 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0514c-d0ad-434d-92bf-1a65d340d536 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:49.541526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0514c-d0ad-434d-92bf-1a65d340d536 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:49.541804 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0514c-d0ad-434d-92bf-1a65d340d536 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:49.553187 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0514c-d0ad-434d-92bf-1a65d340d536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:49.554695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0514c-d0ad-434d-92bf-1a65d340d536 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:49.570985 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63833a88-7b9c-4e0b-a0f8-32e4a34edc49 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:49.573302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-5ee0514c-d0ad-434d-92bf-1a65d340d536 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:49.573302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:49.573302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:49.573302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:49.573302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:49.573302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:49.573302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:49.573302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:49.573302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:49.573302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:49.573302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:49.573302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:49.573302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:25:49.573302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:25:49.573302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:25:49.574160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:25:49.574160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:49.574160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:49.574160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:49.574160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:49.574160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:49.574160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:49.574160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:49.574160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:49.574160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:49.574160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:49.574160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:49.574160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:49.574160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:49.574160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:49.574160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:49.574160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:49.575067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:25:49.575067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:25:49.575067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:49.575067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:49.575067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 223/464] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:49.575710 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f5198-d8ce-4935-a87a-00c5a800af3a None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:49.576016 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f5198-d8ce-4935-a87a-00c5a800af3a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:49.576937 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f5198-d8ce-4935-a87a-00c5a800af3a None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:49.578337 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 224/465] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:49.579049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f5198-d8ce-4935-a87a-00c5a800af3a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:49.604448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:49.610735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0cc5b0-2fa2-4be9-8a75-ff4741ec9464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:49.611588 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cc5b0-2fa2-4be9-8a75-ff4741ec9464 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:49.611712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cc5b0-2fa2-4be9-8a75-ff4741ec9464 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:49.611987 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cc5b0-2fa2-4be9-8a75-ff4741ec9464 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:49.612709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0cc5b0-2fa2-4be9-8a75-ff4741ec9464 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:49.613017 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0cc5b0-2fa2-4be9-8a75-ff4741ec9464 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:49.614404 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0cc5b0-2fa2-4be9-8a75-ff4741ec9464 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:49.617073 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:49.627668 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 242/466] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:49.628377 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0cc5b0-2fa2-4be9-8a75-ff4741ec9464 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:49.690653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f5198-d8ce-4935-a87a-00c5a800af3a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:49.726606 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146da67-5453-455f-a5a1-656767f80a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:49.727271 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146da67-5453-455f-a5a1-656767f80a31 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:49.727548 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146da67-5453-455f-a5a1-656767f80a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:49.727817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146da67-5453-455f-a5a1-656767f80a31 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:49.728532 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146da67-5453-455f-a5a1-656767f80a31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:49.728835 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146da67-5453-455f-a5a1-656767f80a31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:49.730200 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146da67-5453-455f-a5a1-656767f80a31 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:49.731875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0cc5b0-2fa2-4be9-8a75-ff4741ec9464 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:49.752404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 225/467] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:49.752404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645be57-443a-41ca-9b1f-b16290c4459c None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:49.752404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645be57-443a-41ca-9b1f-b16290c4459c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:49.752404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645be57-443a-41ca-9b1f-b16290c4459c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:49.768379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-0645be57-443a-41ca-9b1f-b16290c4459c None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:49.769539 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-0645be57-443a-41ca-9b1f-b16290c4459c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:49.775224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664e717-a9c2-404a-ac47-e5489dc79615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:49.775891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664e717-a9c2-404a-ac47-e5489dc79615 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:49.776171 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664e717-a9c2-404a-ac47-e5489dc79615 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:49.776445 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664e717-a9c2-404a-ac47-e5489dc79615 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:49.777666 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664e717-a9c2-404a-ac47-e5489dc79615 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:49.777978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664e717-a9c2-404a-ac47-e5489dc79615 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:49.779799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664e717-a9c2-404a-ac47-e5489dc79615 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:49.797091 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 243/468] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:49.797091 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664e717-a9c2-404a-ac47-e5489dc79615 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:49.844325 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 226/469] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:49.845220 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645be57-443a-41ca-9b1f-b16290c4459c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:49.864587 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:49.868977 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:49.914434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664e717-a9c2-404a-ac47-e5489dc79615 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:49.951786 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4634b-6a1a-4227-9d48-0d16f50c01df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:49.952374 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4634b-6a1a-4227-9d48-0d16f50c01df None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:49.952660 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4634b-6a1a-4227-9d48-0d16f50c01df None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:49.952936 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4634b-6a1a-4227-9d48-0d16f50c01df None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:49.953820 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4634b-6a1a-4227-9d48-0d16f50c01df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:49.955344 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4634b-6a1a-4227-9d48-0d16f50c01df None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:49.965686 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645be57-443a-41ca-9b1f-b16290c4459c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:50.001880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b96294-cacf-42aa-a1c1-969638dcbbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:50.002559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b96294-cacf-42aa-a1c1-969638dcbbc2 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:50.002852 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b96294-cacf-42aa-a1c1-969638dcbbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:50.003136 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b96294-cacf-42aa-a1c1-969638dcbbc2 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:50.003831 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b96294-cacf-42aa-a1c1-969638dcbbc2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:50.005308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b96294-cacf-42aa-a1c1-969638dcbbc2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:50.046263 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 244/470] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 23 11:25:49 2022] POST /identity/v3/users => generated 246 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:25:50.047187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4634b-6a1a-4227-9d48-0d16f50c01df None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:50.103581 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 227/471] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 23 11:25:49 2022] POST /identity/v3/users => generated 248 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:25:50.111816 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b96294-cacf-42aa-a1c1-969638dcbbc2 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:50.197678 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4634b-6a1a-4227-9d48-0d16f50c01df None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:50.215262 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:50.241434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f76cb0-5b2f-49b4-b149-3c851ed99863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:50.242091 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:50.242645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f76cb0-5b2f-49b4-b149-3c851ed99863 None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:50.242997 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f76cb0-5b2f-49b4-b149-3c851ed99863 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:50.243374 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f76cb0-5b2f-49b4-b149-3c851ed99863 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:50.244314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f76cb0-5b2f-49b4-b149-3c851ed99863 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:50.246301 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f76cb0-5b2f-49b4-b149-3c851ed99863 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:50.287563 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b96294-cacf-42aa-a1c1-969638dcbbc2 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:50.329376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9135799-ef77-4807-b8f7-032aab1b94fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:50.330296 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9135799-ef77-4807-b8f7-032aab1b94fc None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:50.330636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9135799-ef77-4807-b8f7-032aab1b94fc None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:50.330914 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9135799-ef77-4807-b8f7-032aab1b94fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:50.345129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9135799-ef77-4807-b8f7-032aab1b94fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:50.346745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9135799-ef77-4807-b8f7-032aab1b94fc None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:50.349328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 245/472] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 23 11:25:50 2022] POST /identity/v3/users => generated 248 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:25:50.351542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585934a-8e25-4087-b37f-00497baa169e None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:50.351950 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585934a-8e25-4087-b37f-00497baa169e None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:50.352335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585934a-8e25-4087-b37f-00497baa169e None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:50.353839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 246/473] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:50.360678 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263a3d5-fbeb-4d37-a63f-d8053c0f5901 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:50.361124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263a3d5-fbeb-4d37-a63f-d8053c0f5901 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:50.361850 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263a3d5-fbeb-4d37-a63f-d8053c0f5901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:50.370589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-e9135799-ef77-4807-b8f7-032aab1b94fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:50.370589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:50.370589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:50.370589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:50.370589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:50.370589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:50.370589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:50.370589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:50.370589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:50.370589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:50.370589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:50.370589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:50.370589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:25:50.370589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:25:50.370589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:25:50.388218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:25:50.388218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:50.388218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:50.388218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:50.388218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:50.388218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:50.388218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:50.388218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:50.388218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:50.388218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:50.388218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:50.388218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:50.388218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:50.388218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:50.388218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:50.388218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:50.388218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:50.390401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:25:50.390401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:25:50.390401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:50.390401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:50.390401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 228/474] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:50.390401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9135799-ef77-4807-b8f7-032aab1b94fc None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:50.397171 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-b263a3d5-fbeb-4d37-a63f-d8053c0f5901 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:50.399388 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b263a3d5-fbeb-4d37-a63f-d8053c0f5901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:50.518801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9135799-ef77-4807-b8f7-032aab1b94fc None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:50.523596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 247/475] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:50.549331 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:50.559100 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263a3d5-fbeb-4d37-a63f-d8053c0f5901 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:50.571809 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d3041-0918-4b80-a6e8-7d143a3a50ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:50.572491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d3041-0918-4b80-a6e8-7d143a3a50ea None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:50.572830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d3041-0918-4b80-a6e8-7d143a3a50ea None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:50.573114 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d3041-0918-4b80-a6e8-7d143a3a50ea None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:50.573847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d3041-0918-4b80-a6e8-7d143a3a50ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:50.574160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d3041-0918-4b80-a6e8-7d143a3a50ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:50.578965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d3041-0918-4b80-a6e8-7d143a3a50ea None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:50.597273 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 229/476] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:50.606485 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d3041-0918-4b80-a6e8-7d143a3a50ea None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:50.689730 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:50.695619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263a3d5-fbeb-4d37-a63f-d8053c0f5901 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:50.720568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d3041-0918-4b80-a6e8-7d143a3a50ea None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:50.736432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f753e1b-cc21-4424-b307-3a74e2aabcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:50.737111 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f753e1b-cc21-4424-b307-3a74e2aabcf2 None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:50.737400 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f753e1b-cc21-4424-b307-3a74e2aabcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:50.737680 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f753e1b-cc21-4424-b307-3a74e2aabcf2 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:50.738457 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f753e1b-cc21-4424-b307-3a74e2aabcf2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:50.741536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f753e1b-cc21-4424-b307-3a74e2aabcf2 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:50.765439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227108d-e524-4ce4-ae6d-100f3cf61df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:50.766122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227108d-e524-4ce4-ae6d-100f3cf61df6 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:50.767123 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227108d-e524-4ce4-ae6d-100f3cf61df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:50.767123 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227108d-e524-4ce4-ae6d-100f3cf61df6 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:50.767619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8227108d-e524-4ce4-ae6d-100f3cf61df6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:50.769075 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8227108d-e524-4ce4-ae6d-100f3cf61df6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:50.791730 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.sql.core [None req-0f753e1b-cc21-4424-b307-3a74e2aabcf2 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 11:25:50.791730 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 11:25:50.791730 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [parameters: {'id': '78fb0aae66f64a808c31dddc23084ca9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 11:25:50.791730 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80255) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 11:25:50.809336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-0f753e1b-cc21-4424-b307-3a74e2aabcf2 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 11:25:50.809336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:50.809336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 11:25:50.809336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 11:25:50.809336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 11:25:50.809336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 11:25:50.809336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 11:25:50.809336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 11:25:50.809336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 11:25:50.809336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application conn.query(q) Jun 23 11:25:50.809336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 11:25:50.809336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 11:25:50.809336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 11:25:50.809336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application result.read() Jun 23 11:25:50.809336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 11:25:50.810494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 11:25:50.810494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 11:25:50.810494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 11:25:50.810494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 11:25:50.810494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 11:25:50.810494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 11:25:50.810494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 11:25:50.810494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 11:25:50.810494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:50.810494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 11:25:50.810494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:50.810494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:50.810494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 11:25:50.810494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 11:25:50.810494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 23 11:25:50.810494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return ref.to_dict() Jun 23 11:25:50.810494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 11:25:50.810494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application next(self.gen) Jun 23 11:25:50.812155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 11:25:50.812155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application yield resource Jun 23 11:25:50.812155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 11:25:50.812155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application next(self.gen) Jun 23 11:25:50.812155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 11:25:50.812155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 11:25:50.812155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 11:25:50.812155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 11:25:50.812155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 11:25:50.812155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise self.value Jun 23 11:25:50.812155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 11:25:50.812155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 11:25:50.812155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 11:25:50.812155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application session.commit() Jun 23 11:25:50.812155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 11:25:50.812155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 11:25:50.812155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 11:25:50.813024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 11:25:50.813024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 11:25:50.813024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self.session.flush() Jun 23 11:25:50.813024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 11:25:50.813024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 11:25:50.813024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 11:25:50.813024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 11:25:50.813024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 11:25:50.813024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application compat.raise_( Jun 23 11:25:50.813024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 11:25:50.813024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception Jun 23 11:25:50.813024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 11:25:50.813024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 11:25:50.813024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 11:25:50.813024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 11:25:50.813024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 11:25:50.813024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 11:25:50.813907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 11:25:50.813907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 11:25:50.813907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 11:25:50.813907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 11:25:50.813907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 11:25:50.813907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 11:25:50.813907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 11:25:50.813907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 11:25:50.813907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 11:25:50.813907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 11:25:50.813907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 11:25:50.813907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 11:25:50.813907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 11:25:50.813907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 11:25:50.813907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 11:25:50.813907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception Jun 23 11:25:50.815873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 11:25:50.815873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 11:25:50.815873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 11:25:50.815873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 11:25:50.815873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 11:25:50.815873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 11:25:50.815873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 11:25:50.815873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application conn.query(q) Jun 23 11:25:50.815873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 11:25:50.815873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 11:25:50.815873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 11:25:50.815873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application result.read() Jun 23 11:25:50.815873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 11:25:50.815873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 11:25:50.815873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 11:25:50.815873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 11:25:50.818051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 11:25:50.818051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 11:25:50.818051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 11:25:50.818051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 11:25:50.818051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 11:25:50.818051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 11:25:50.818051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [parameters: {'id': '78fb0aae66f64a808c31dddc23084ca9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 11:25:50.818051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 11:25:50.818051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:50.818051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:25:50.818051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:50.818051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:50.818051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:50.818051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:50.818051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:50.818051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:50.820176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:50.820176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:50.820176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:50.820176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:50.820176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:50.820176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:50.820176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 23 11:25:50.820176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 23 11:25:50.820176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:50.820176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:50.820176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 23 11:25:50.820176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 23 11:25:50.820176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 11:25:50.820176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 11:25:50.820176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 11:25:50.820176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:50.822396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 248/477] 10.222.0.13 () {60 vars in 1131 bytes} [Thu Jun 23 11:25:50 2022] POST /identity/v3/roles => generated 144 bytes in 253 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:25:50.822396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f753e1b-cc21-4424-b307-3a74e2aabcf2 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:50.862987 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 230/478] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 23 11:25:50 2022] POST /identity/v3/users => generated 249 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:25:50.972306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f753e1b-cc21-4424-b307-3a74e2aabcf2 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:51.016878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8a286-0a6b-4695-a1af-ed50153f0549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:51.018143 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8a286-0a6b-4695-a1af-ed50153f0549 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:51.018655 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8a286-0a6b-4695-a1af-ed50153f0549 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:51.019229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8a286-0a6b-4695-a1af-ed50153f0549 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:51.033885 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8a286-0a6b-4695-a1af-ed50153f0549 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:51.042477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-8fd8a286-0a6b-4695-a1af-ed50153f0549 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:51.042477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:51.042477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:51.042477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:51.042477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:51.042477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:51.042477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:51.042477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:51.042477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:51.042477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:51.042477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:51.042477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:51.042477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:51.042477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:51.042477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:51.042477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:51.043355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:51.043355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:51.043355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:51.043355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:51.043355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:51.043355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:51.043355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:51.043355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:51.043355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:51.043355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:51.043355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:51.043355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:51.043355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:51.043355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:51.043355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:51.043355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:51.044168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:51.044168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:51.044168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:51.044168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:51.044168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:51.044168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:51.044168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:51.044168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:51.044168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:51.044168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:51.044168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:25:51.044168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:51.047412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 249/479] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:25:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:51.054552 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227108d-e524-4ce4-ae6d-100f3cf61df6 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:51.095139 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:51.166831 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227108d-e524-4ce4-ae6d-100f3cf61df6 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:51.208679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346814c-723c-4061-b51c-410f193372ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:51.209369 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346814c-723c-4061-b51c-410f193372ce None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:51.209657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346814c-723c-4061-b51c-410f193372ce None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:51.209933 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346814c-723c-4061-b51c-410f193372ce None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:51.210722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1346814c-723c-4061-b51c-410f193372ce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:51.211036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1346814c-723c-4061-b51c-410f193372ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:51.212416 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1346814c-723c-4061-b51c-410f193372ce None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:51.222938 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 231/480] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:25:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:52.493383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a95e88-3862-45f9-991c-b3487065a3ce None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:52.493796 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a95e88-3862-45f9-991c-b3487065a3ce None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:52.493975 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a95e88-3862-45f9-991c-b3487065a3ce None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:52.495667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 250/481] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:52.501098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16023030-cf75-405c-b503-ce546139610a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:52.501525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16023030-cf75-405c-b503-ce546139610a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:52.501696 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16023030-cf75-405c-b503-ce546139610a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:52.527874 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-16023030-cf75-405c-b503-ce546139610a None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:52.529636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-16023030-cf75-405c-b503-ce546139610a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:52.620834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 232/482] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:52.661107 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a95e88-3862-45f9-991c-b3487065a3ce None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:52.693551 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:52.722210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0c46a-e5b5-4a1d-b96e-6fe9d3062d03 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:52.722515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0c46a-e5b5-4a1d-b96e-6fe9d3062d03 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:52.722792 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0c46a-e5b5-4a1d-b96e-6fe9d3062d03 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:52.724180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 233/483] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:25:52.729375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904a393-e0d7-4663-8081-49caa5701e5c None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:52.729669 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904a393-e0d7-4663-8081-49caa5701e5c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:52.729959 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904a393-e0d7-4663-8081-49caa5701e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:52.750716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-7904a393-e0d7-4663-8081-49caa5701e5c None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:52.752242 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7904a393-e0d7-4663-8081-49caa5701e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:52.778934 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:52.813226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a95e88-3862-45f9-991c-b3487065a3ce None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:52.886836 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 234/484] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:25:52.889812 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa436991-60ce-46b4-ab3a-3eeded170dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:52.891688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa436991-60ce-46b4-ab3a-3eeded170dbd None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:52.891688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa436991-60ce-46b4-ab3a-3eeded170dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:52.891688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa436991-60ce-46b4-ab3a-3eeded170dbd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:52.902292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa436991-60ce-46b4-ab3a-3eeded170dbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:52.910723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-aa436991-60ce-46b4-ab3a-3eeded170dbd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:25:52.910723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:52.910723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:52.910723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:52.910723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:52.910723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:52.910723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:52.910723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:52.910723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:52.910723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:52.910723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:52.910723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:52.910723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:52.910723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:52.910723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:52.910723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:52.911770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:52.911770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:52.911770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:52.911770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:52.911770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:52.911770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:52.911770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:52.911770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:52.911770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:52.911770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:52.911770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:52.911770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:52.911770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:52.911770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:52.911770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:52.911770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:52.912749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:52.912749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:52.912749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:52.912749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:52.912749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:52.912749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:52.912749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:52.912749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:52.912749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:52.912749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:52.912749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:25:52.912749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:52.915099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 251/485] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:25:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:52.920983 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7904a393-e0d7-4663-8081-49caa5701e5c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:52.925179 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa436991-60ce-46b4-ab3a-3eeded170dbd None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:53.052253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7904a393-e0d7-4663-8081-49caa5701e5c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:53.086932 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:53.107980 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa436991-60ce-46b4-ab3a-3eeded170dbd None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:53.127104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f2138-710b-4cf8-b39b-8dc190f752cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:53.127782 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f2138-710b-4cf8-b39b-8dc190f752cf None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:53.128067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f2138-710b-4cf8-b39b-8dc190f752cf None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:53.128346 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f2138-710b-4cf8-b39b-8dc190f752cf None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:53.129100 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f2138-710b-4cf8-b39b-8dc190f752cf None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:53.129410 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f2138-710b-4cf8-b39b-8dc190f752cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:53.130875 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f2138-710b-4cf8-b39b-8dc190f752cf None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:53.139864 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 235/486] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:25:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:53.142271 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84778d-3c9d-4006-a03d-056537edde71 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:53.142586 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84778d-3c9d-4006-a03d-056537edde71 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:53.142869 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84778d-3c9d-4006-a03d-056537edde71 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:53.144239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 236/487] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:53.148843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84778d-3c9d-4006-a03d-056537edde71 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:53.149629 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d3b7b-00a7-4fff-a4ff-d872930dbdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:53.150431 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d3b7b-00a7-4fff-a4ff-d872930dbdc6 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:53.150745 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d3b7b-00a7-4fff-a4ff-d872930dbdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:53.151021 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d3b7b-00a7-4fff-a4ff-d872930dbdc6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:53.165956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d3b7b-00a7-4fff-a4ff-d872930dbdc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:53.169266 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:53.176762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-9f1d3b7b-00a7-4fff-a4ff-d872930dbdc6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:25:53.176762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:53.176762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:53.176762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:53.176762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:53.176762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:53.176762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:53.176762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:53.176762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:53.176762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:53.176762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:53.176762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:53.176762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:53.176762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:53.176762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:53.176762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:53.179562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:53.179562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:53.179562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:53.179562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:53.179562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:53.179562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:53.179562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:53.179562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:53.179562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:53.179562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:53.179562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:53.179562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:53.179562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:53.179562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:53.179562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:53.179562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:53.180354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:53.180354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:53.180354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:53.180354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:53.180354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:53.180354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:53.180354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:53.180354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:53.180354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:53.180354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:53.180354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:25:53.180354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:53.180354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 252/488] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:25:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:25:53.180354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b986f7-8797-4fa5-8408-38b01cb8ee9e None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:53.181089 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b986f7-8797-4fa5-8408-38b01cb8ee9e None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:53.181089 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b986f7-8797-4fa5-8408-38b01cb8ee9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:53.202587 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-82b986f7-8797-4fa5-8408-38b01cb8ee9e None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:53.204205 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-82b986f7-8797-4fa5-8408-38b01cb8ee9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:53.282147 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 253/489] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:53.283394 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b986f7-8797-4fa5-8408-38b01cb8ee9e None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:53.284696 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84778d-3c9d-4006-a03d-056537edde71 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:53.324371 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da7210-8fa0-4df7-8f08-001a86233eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:53.325287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da7210-8fa0-4df7-8f08-001a86233eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:53.325675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da7210-8fa0-4df7-8f08-001a86233eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:53.326503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da7210-8fa0-4df7-8f08-001a86233eb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:53.338133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da7210-8fa0-4df7-8f08-001a86233eb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:53.339669 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da7210-8fa0-4df7-8f08-001a86233eb6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:53.358808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-49da7210-8fa0-4df7-8f08-001a86233eb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:53.358808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:53.358808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:53.358808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:53.358808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:53.358808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:53.358808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:53.358808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:53.358808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:53.358808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:53.358808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:53.358808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:53.358808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:25:53.358808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:25:53.358808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:25:53.359733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:25:53.359733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:53.359733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:53.359733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:53.359733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:53.359733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:53.359733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:53.359733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:53.359733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:53.359733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:53.359733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:53.359733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:53.359733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:53.359733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:53.359733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:53.359733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:53.359733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:53.360560 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:25:53.360560 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:25:53.360560 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:53.360560 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:53.360560 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 237/490] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:53.361249 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da7210-8fa0-4df7-8f08-001a86233eb6 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:53.377746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:53.409786 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b986f7-8797-4fa5-8408-38b01cb8ee9e None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:53.439376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:53.445771 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def7e86-3c34-42e2-b745-6332d22e18b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:53.446487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def7e86-3c34-42e2-b745-6332d22e18b3 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:53.446803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def7e86-3c34-42e2-b745-6332d22e18b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:53.447077 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def7e86-3c34-42e2-b745-6332d22e18b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:53.453458 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def7e86-3c34-42e2-b745-6332d22e18b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:53.453458 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def7e86-3c34-42e2-b745-6332d22e18b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:53.453458 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def7e86-3c34-42e2-b745-6332d22e18b3 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:53.459181 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 254/491] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:25:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:53.460014 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def7e86-3c34-42e2-b745-6332d22e18b3 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:53.474472 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da7210-8fa0-4df7-8f08-001a86233eb6 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:53.511426 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5635909-aa3a-4d60-a29a-f832a23b939a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:53.512239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5635909-aa3a-4d60-a29a-f832a23b939a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:53.512525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5635909-aa3a-4d60-a29a-f832a23b939a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:53.512803 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5635909-aa3a-4d60-a29a-f832a23b939a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:53.524768 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5635909-aa3a-4d60-a29a-f832a23b939a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:53.538897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-e5635909-aa3a-4d60-a29a-f832a23b939a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:25:53.538897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:53.538897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:53.538897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:53.538897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:53.538897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:53.538897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:53.538897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:53.538897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:53.538897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:53.538897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:53.538897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:53.538897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:53.538897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:53.538897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:53.538897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:53.539813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:53.539813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:53.539813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:53.539813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:53.539813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:53.539813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:53.539813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:53.539813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:53.539813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:53.539813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:53.539813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:53.539813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:53.539813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:53.539813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:53.539813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:53.539813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:53.540596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:53.540596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:53.540596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:53.540596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:53.540596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:53.540596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:53.540596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:53.540596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:53.540596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:53.540596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:53.540596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:25:53.540596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:53.543507 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 238/492] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:25:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:53.544406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5635909-aa3a-4d60-a29a-f832a23b939a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:53.587274 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def7e86-3c34-42e2-b745-6332d22e18b3 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:53.625102 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b3486-0000-4dae-8c75-18d6ccda44d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:53.625770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b3486-0000-4dae-8c75-18d6ccda44d0 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:53.626052 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b3486-0000-4dae-8c75-18d6ccda44d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:53.626331 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b3486-0000-4dae-8c75-18d6ccda44d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:53.627113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b3486-0000-4dae-8c75-18d6ccda44d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:53.627422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b3486-0000-4dae-8c75-18d6ccda44d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:53.628938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b3486-0000-4dae-8c75-18d6ccda44d0 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:53.630661 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:53.643772 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 255/493] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:53.644381 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b3486-0000-4dae-8c75-18d6ccda44d0 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:53.655988 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5635909-aa3a-4d60-a29a-f832a23b939a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:53.687800 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:53.693755 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102fc81-6775-44ee-bbd2-2cf4da4a1b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:53.694704 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102fc81-6775-44ee-bbd2-2cf4da4a1b85 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:53.694993 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102fc81-6775-44ee-bbd2-2cf4da4a1b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:53.699563 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102fc81-6775-44ee-bbd2-2cf4da4a1b85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:53.707076 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102fc81-6775-44ee-bbd2-2cf4da4a1b85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:53.708668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102fc81-6775-44ee-bbd2-2cf4da4a1b85 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:53.727995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-5102fc81-6775-44ee-bbd2-2cf4da4a1b85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:53.727995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:53.727995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:53.727995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:53.727995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:53.727995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:53.727995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:53.727995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:53.727995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:53.727995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:53.727995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:53.727995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:53.727995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:25:53.727995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:25:53.727995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:25:53.731361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:25:53.731361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:53.731361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:53.731361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:53.731361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:53.731361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:53.731361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:53.731361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:53.731361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:53.731361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:53.731361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:53.731361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:53.731361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:53.731361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:53.731361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:53.731361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:53.731361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:53.732743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:25:53.732743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:25:53.732743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:53.732743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:53.732743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 239/494] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:53.732743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102fc81-6775-44ee-bbd2-2cf4da4a1b85 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:53.756236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b3486-0000-4dae-8c75-18d6ccda44d0 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:53.792964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ba9f2-9c1c-4684-95ab-7d9719534f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:53.793628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ba9f2-9c1c-4684-95ab-7d9719534f4f None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:53.793931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ba9f2-9c1c-4684-95ab-7d9719534f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:53.794180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ba9f2-9c1c-4684-95ab-7d9719534f4f None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:53.795624 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ba9f2-9c1c-4684-95ab-7d9719534f4f None admin] RBAC: Adding query filter params (name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:53.795923 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ba9f2-9c1c-4684-95ab-7d9719534f4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:53.797289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ba9f2-9c1c-4684-95ab-7d9719534f4f None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:53.806277 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 256/495] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:25:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:53.806277 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3d942-293c-4ecf-a99f-2a83cd48d1df None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:53.806277 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3d942-293c-4ecf-a99f-2a83cd48d1df None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:53.806277 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3d942-293c-4ecf-a99f-2a83cd48d1df None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:53.811786 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 257/496] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:53.812319 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3d942-293c-4ecf-a99f-2a83cd48d1df None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:53.851001 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102fc81-6775-44ee-bbd2-2cf4da4a1b85 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:53.878604 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:53.897024 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3d22b-0606-42bd-8761-88ccd8d678cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:53.897817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3d22b-0606-42bd-8761-88ccd8d678cd None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:53.898105 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3d22b-0606-42bd-8761-88ccd8d678cd None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:53.898380 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3d22b-0606-42bd-8761-88ccd8d678cd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:53.906280 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3d22b-0606-42bd-8761-88ccd8d678cd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:53.907762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3d22b-0606-42bd-8761-88ccd8d678cd None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:53.923528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-47d3d22b-0606-42bd-8761-88ccd8d678cd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:25:53.923528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:53.923528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:53.923528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:53.923528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:53.923528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:53.923528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:53.923528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:53.923528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:53.923528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:53.923528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:53.923528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:53.923528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:25:53.923528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:25:53.923528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:25:53.923528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:25:53.924634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:53.924634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:53.924634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:25:53.924634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:25:53.924634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:25:53.924634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:25:53.924634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:53.924634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:53.924634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:53.924634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:53.924634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:53.924634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:53.924634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:53.924634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:53.924634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:53.924634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:53.924634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:53.924634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:53.925533 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:53.925533 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:53.925533 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:25:53.925533 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:25:53.925533 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:25:53.925533 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:25:53.925533 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:25:53.925533 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:25:53.925533 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:25:53.925533 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:53.926034 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3d942-293c-4ecf-a99f-2a83cd48d1df None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:53.926111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 240/497] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jun 23 11:25:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:53.926111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3d22b-0606-42bd-8761-88ccd8d678cd None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:53.962217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854111b8-a401-412e-b667-967b0e7507f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:53.962969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854111b8-a401-412e-b667-967b0e7507f9 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:53.963250 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854111b8-a401-412e-b667-967b0e7507f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:53.963576 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854111b8-a401-412e-b667-967b0e7507f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:53.964324 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854111b8-a401-412e-b667-967b0e7507f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:53.964627 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854111b8-a401-412e-b667-967b0e7507f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:53.965983 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854111b8-a401-412e-b667-967b0e7507f9 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:53.970792 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:53.977747 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 258/498] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:53.979033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa6fc9-416a-4cf9-aefd-3916898022f0 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:53.979323 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa6fc9-416a-4cf9-aefd-3916898022f0 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:53.979596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa6fc9-416a-4cf9-aefd-3916898022f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:53.996284 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-0daa6fc9-416a-4cf9-aefd-3916898022f0 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:53.997468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-0daa6fc9-416a-4cf9-aefd-3916898022f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:54.027306 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3d22b-0606-42bd-8761-88ccd8d678cd None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:54.061850 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d25a8-947b-4d55-885a-5d509ac71c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:54.062666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d25a8-947b-4d55-885a-5d509ac71c3d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:54.062951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d25a8-947b-4d55-885a-5d509ac71c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:54.063224 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d25a8-947b-4d55-885a-5d509ac71c3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:54.064178 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 259/499] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:54.065476 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe06304-065e-4916-889c-207e9cfac39e None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:54.065773 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe06304-065e-4916-889c-207e9cfac39e None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:54.066053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe06304-065e-4916-889c-207e9cfac39e None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:54.067483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 260/500] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:54.068286 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe06304-065e-4916-889c-207e9cfac39e None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:54.085468 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d25a8-947b-4d55-885a-5d509ac71c3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:54.086994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d25a8-947b-4d55-885a-5d509ac71c3d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:54.104517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-c68d25a8-947b-4d55-885a-5d509ac71c3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:54.104517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:54.104517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:54.104517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:54.104517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:54.104517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:54.104517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:54.104517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:54.104517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:54.104517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:54.104517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:54.104517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:54.104517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:25:54.104517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:25:54.104517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:25:54.105375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:25:54.105375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:54.105375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:54.105375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:54.105375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:54.105375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:54.105375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:54.105375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:54.105375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:54.105375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:54.105375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:54.105375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:54.105375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:54.105375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:54.105375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:54.105375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:54.105375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:54.106225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:25:54.106225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:25:54.106225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:54.106225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:54.106225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 241/501] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:54.106495 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d25a8-947b-4d55-885a-5d509ac71c3d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:54.125786 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:54.179386 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe06304-065e-4916-889c-207e9cfac39e None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:54.202733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:54.209226 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d25a8-947b-4d55-885a-5d509ac71c3d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:54.212485 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56832934-9edc-40d2-a6e7-5a1e3fd88f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:54.213150 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56832934-9edc-40d2-a6e7-5a1e3fd88f15 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:54.213434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56832934-9edc-40d2-a6e7-5a1e3fd88f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:54.217018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56832934-9edc-40d2-a6e7-5a1e3fd88f15 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:54.217018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56832934-9edc-40d2-a6e7-5a1e3fd88f15 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:54.217018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56832934-9edc-40d2-a6e7-5a1e3fd88f15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:54.217018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56832934-9edc-40d2-a6e7-5a1e3fd88f15 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:54.243975 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76feb151-ad1d-47d6-9df0-e112824089f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:54.244765 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76feb151-ad1d-47d6-9df0-e112824089f0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:54.245056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76feb151-ad1d-47d6-9df0-e112824089f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:54.245333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76feb151-ad1d-47d6-9df0-e112824089f0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:54.252533 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76feb151-ad1d-47d6-9df0-e112824089f0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:54.253153 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 261/502] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jun 23 11:25:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:54.253991 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76feb151-ad1d-47d6-9df0-e112824089f0 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:54.254479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0584ae-1957-4fec-8f4f-39b6d77c57cb None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:54.254804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0584ae-1957-4fec-8f4f-39b6d77c57cb None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:54.255084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0584ae-1957-4fec-8f4f-39b6d77c57cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:54.270636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-76feb151-ad1d-47d6-9df0-e112824089f0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 11:25:54.270636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:54.270636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:54.270636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:54.270636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:54.270636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:54.270636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:54.270636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:54.270636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:54.270636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:54.270636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:54.270636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:54.270636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:25:54.270636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:25:54.270636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:25:54.270636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:25:54.271700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:54.271700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:54.271700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:25:54.271700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:25:54.271700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:25:54.271700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:25:54.271700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:54.271700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:54.271700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:54.271700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:54.271700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:54.271700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:54.271700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:54.271700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:54.271700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:54.271700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:54.271700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:54.271700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:54.272623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:54.272623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:54.272623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:25:54.272623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:25:54.272623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:25:54.272623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:25:54.272623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:25:54.272623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:25:54.272623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 11:25:54.272623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:54.272623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 242/503] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:54.273175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-3c0584ae-1957-4fec-8f4f-39b6d77c57cb None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:54.273244 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76feb151-ad1d-47d6-9df0-e112824089f0 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:54.273508 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-3c0584ae-1957-4fec-8f4f-39b6d77c57cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:54.339495 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 262/504] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:54.340309 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0584ae-1957-4fec-8f4f-39b6d77c57cb None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:54.377315 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:54.393682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76feb151-ad1d-47d6-9df0-e112824089f0 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:54.427246 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c42c13-0674-4ad2-b060-4a1bd9cc488b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:54.428038 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c42c13-0674-4ad2-b060-4a1bd9cc488b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:54.428321 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c42c13-0674-4ad2-b060-4a1bd9cc488b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:54.428608 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c42c13-0674-4ad2-b060-4a1bd9cc488b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:54.439220 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c42c13-0674-4ad2-b060-4a1bd9cc488b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:54.444219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0584ae-1957-4fec-8f4f-39b6d77c57cb None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:54.447232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-70c42c13-0674-4ad2-b060-4a1bd9cc488b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:25:54.447232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:54.447232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:54.447232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:54.447232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:54.447232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:54.447232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:54.447232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:54.447232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:54.447232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:54.447232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:54.447232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:54.447232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:54.447232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:54.447232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:54.447232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:54.448256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:54.448256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:54.448256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:54.448256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:54.448256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:54.448256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:54.448256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:54.448256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:54.448256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:54.448256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:54.448256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:54.448256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:54.448256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:54.448256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:54.448256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:54.448256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:54.449086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:54.449086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:54.449086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:54.449086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:54.449086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:54.449086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:54.449086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:54.449086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:54.449086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:54.449086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:54.449086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:25:54.449086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:54.449086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 243/505] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:25:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:54.449739 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c42c13-0674-4ad2-b060-4a1bd9cc488b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:54.465647 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:54.479247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ead2e-1c39-4311-9e3f-32b764e367ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:54.479904 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ead2e-1c39-4311-9e3f-32b764e367ba None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:54.480184 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ead2e-1c39-4311-9e3f-32b764e367ba None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:54.480455 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ead2e-1c39-4311-9e3f-32b764e367ba None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:54.481173 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ead2e-1c39-4311-9e3f-32b764e367ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:54.481478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ead2e-1c39-4311-9e3f-32b764e367ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:54.483418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ead2e-1c39-4311-9e3f-32b764e367ba None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:54.495038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 263/506] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:54.496013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ead2e-1c39-4311-9e3f-32b764e367ba None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:54.551841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c42c13-0674-4ad2-b060-4a1bd9cc488b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:54.586200 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a10d72-b94a-44b4-9ffe-d1bb7e4adb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:54.586882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a10d72-b94a-44b4-9ffe-d1bb7e4adb81 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:54.587163 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a10d72-b94a-44b4-9ffe-d1bb7e4adb81 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:54.587439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a10d72-b94a-44b4-9ffe-d1bb7e4adb81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:54.589992 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:54.597357 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ead2e-1c39-4311-9e3f-32b764e367ba None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:54.631899 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c01cb-68f1-42cc-a0a8-b249801b74ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:54.632525 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c01cb-68f1-42cc-a0a8-b249801b74ba None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:54.632803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c01cb-68f1-42cc-a0a8-b249801b74ba None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:54.633075 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c01cb-68f1-42cc-a0a8-b249801b74ba None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:54.633900 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c01cb-68f1-42cc-a0a8-b249801b74ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:54.634207 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c01cb-68f1-42cc-a0a8-b249801b74ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:54.635664 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c01cb-68f1-42cc-a0a8-b249801b74ba None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:54.653443 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 264/507] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Jun 23 11:25:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:54.654496 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c01cb-68f1-42cc-a0a8-b249801b74ba None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:54.656640 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a10d72-b94a-44b4-9ffe-d1bb7e4adb81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:54.658092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a10d72-b94a-44b4-9ffe-d1bb7e4adb81 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:54.673135 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:54.755645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c01cb-68f1-42cc-a0a8-b249801b74ba None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:54.757119 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 244/508] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:54.757865 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a10d72-b94a-44b4-9ffe-d1bb7e4adb81 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:54.789351 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3273a-e765-428c-baba-03ada044048e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:54.790142 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3273a-e765-428c-baba-03ada044048e None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:54.790430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3273a-e765-428c-baba-03ada044048e None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:54.790779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3273a-e765-428c-baba-03ada044048e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:54.801337 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3273a-e765-428c-baba-03ada044048e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:54.803032 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3273a-e765-428c-baba-03ada044048e None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:54.821460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-daf3273a-e765-428c-baba-03ada044048e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:54.821460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:54.821460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:54.821460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:54.821460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:54.821460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:54.821460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:54.821460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:54.821460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:54.821460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:54.821460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:54.821460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:54.821460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:25:54.821460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:25:54.821460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:25:54.822472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:25:54.822472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:54.822472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:54.822472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:54.822472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:54.822472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:54.822472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:54.822472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:54.822472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:54.822472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:54.822472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:54.822472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:54.822472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:54.822472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:54.822472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:54.822472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:54.822472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:54.828783 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:25:54.828783 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:25:54.828783 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:54.828783 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:54.828783 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 265/509] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:54.828783 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3273a-e765-428c-baba-03ada044048e None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:54.828783 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:54.864603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a10d72-b94a-44b4-9ffe-d1bb7e4adb81 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:54.878322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:54.897974 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1915e-d7cf-4345-9c60-239454197870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:54.898702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1915e-d7cf-4345-9c60-239454197870 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:54.898990 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1915e-d7cf-4345-9c60-239454197870 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:54.899303 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1915e-d7cf-4345-9c60-239454197870 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:54.900059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1915e-d7cf-4345-9c60-239454197870 None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:54.900362 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1915e-d7cf-4345-9c60-239454197870 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:54.901736 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1915e-d7cf-4345-9c60-239454197870 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:54.910614 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 245/510] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:25:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:54.911401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1915e-d7cf-4345-9c60-239454197870 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:54.927739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3273a-e765-428c-baba-03ada044048e None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:54.960879 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda261f7-c30f-4dfb-afc6-96caedd39ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:54.961671 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda261f7-c30f-4dfb-afc6-96caedd39ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:54.961951 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda261f7-c30f-4dfb-afc6-96caedd39ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:54.962223 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda261f7-c30f-4dfb-afc6-96caedd39ef3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:54.969984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda261f7-c30f-4dfb-afc6-96caedd39ef3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:54.971462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda261f7-c30f-4dfb-afc6-96caedd39ef3 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:54.987125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-dda261f7-c30f-4dfb-afc6-96caedd39ef3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 11:25:54.987125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:54.987125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:54.987125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:54.987125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:54.987125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:54.987125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:54.987125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:54.987125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:54.987125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:54.987125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:54.987125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:54.987125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:25:54.987125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:25:54.987125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:25:54.987125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:25:54.988039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:54.988039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:54.988039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:25:54.988039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:25:54.988039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:25:54.988039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:25:54.988039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:54.988039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:54.988039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:54.988039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:54.988039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:54.988039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:54.988039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:54.988039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:54.988039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:54.988039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:54.988039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:54.988039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:54.988925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:54.988925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:54.988925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:25:54.988925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:25:54.988925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:25:54.988925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:25:54.988925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:25:54.988925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:25:54.988925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 11:25:54.988925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:54.988925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 266/511] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:25:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:54.989475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda261f7-c30f-4dfb-afc6-96caedd39ef3 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:55.013886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1915e-d7cf-4345-9c60-239454197870 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:55.048249 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5271346-6efa-4516-a754-8fd1bcb2acc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:55.048892 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5271346-6efa-4516-a754-8fd1bcb2acc2 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:55.049180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5271346-6efa-4516-a754-8fd1bcb2acc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:55.049451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5271346-6efa-4516-a754-8fd1bcb2acc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:55.052847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:55.090351 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda261f7-c30f-4dfb-afc6-96caedd39ef3 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:55.094131 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:55.117038 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5271346-6efa-4516-a754-8fd1bcb2acc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:55.118489 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5271346-6efa-4516-a754-8fd1bcb2acc2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:55.124174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda50370-9c9a-4b42-bbd6-f7209b2377a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:55.124956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda50370-9c9a-4b42-bbd6-f7209b2377a1 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:55.125239 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda50370-9c9a-4b42-bbd6-f7209b2377a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:55.125514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda50370-9c9a-4b42-bbd6-f7209b2377a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:55.135856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda50370-9c9a-4b42-bbd6-f7209b2377a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:55.137316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda50370-9c9a-4b42-bbd6-f7209b2377a1 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:55.155719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-dda50370-9c9a-4b42-bbd6-f7209b2377a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:55.155719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:55.155719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:55.155719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:55.155719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:55.155719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:55.155719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:55.155719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:55.155719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:55.155719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:55.155719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:55.155719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:55.155719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:25:55.155719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:25:55.155719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:25:55.156542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:25:55.156542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:55.156542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:55.156542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:55.156542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:55.156542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:55.156542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:55.156542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:55.156542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:55.156542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:55.156542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:55.156542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:55.156542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:55.156542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:55.156542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:55.156542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:55.156542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:55.157411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:25:55.157411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:25:55.157411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:55.157411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:55.157411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 267/512] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:55.157664 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda50370-9c9a-4b42-bbd6-f7209b2377a1 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:55.219695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 246/513] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:55.220534 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5271346-6efa-4516-a754-8fd1bcb2acc2 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:55.257391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda50370-9c9a-4b42-bbd6-f7209b2377a1 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:55.273475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:55.291131 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132346c-187c-4c3a-b0a9-a78d8d4a21af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:55.291794 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132346c-187c-4c3a-b0a9-a78d8d4a21af None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:55.292112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132346c-187c-4c3a-b0a9-a78d8d4a21af None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:55.292447 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132346c-187c-4c3a-b0a9-a78d8d4a21af None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:55.293245 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6132346c-187c-4c3a-b0a9-a78d8d4a21af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:55.293568 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6132346c-187c-4c3a-b0a9-a78d8d4a21af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:55.294370 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:55.295023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6132346c-187c-4c3a-b0a9-a78d8d4a21af None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:55.306546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 268/514] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:55.307350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132346c-187c-4c3a-b0a9-a78d8d4a21af None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:55.317632 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5271346-6efa-4516-a754-8fd1bcb2acc2 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:55.351873 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d216a02-36e3-485c-af9c-baad0b4cf825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:55.352662 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d216a02-36e3-485c-af9c-baad0b4cf825 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:55.353210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d216a02-36e3-485c-af9c-baad0b4cf825 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:55.353711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d216a02-36e3-485c-af9c-baad0b4cf825 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:55.364124 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d216a02-36e3-485c-af9c-baad0b4cf825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:55.365572 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d216a02-36e3-485c-af9c-baad0b4cf825 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:55.382859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-4d216a02-36e3-485c-af9c-baad0b4cf825 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:55.382859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:55.382859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:55.382859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:55.382859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:55.382859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:55.382859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:55.382859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:55.382859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:55.382859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:55.382859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:55.382859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:55.382859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:25:55.382859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:25:55.382859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:25:55.383747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:25:55.383747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:55.383747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:55.383747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:55.383747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:55.383747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:55.383747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:55.383747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:55.383747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:55.383747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:55.383747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:55.383747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:55.383747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:55.383747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:55.383747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:55.383747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:55.383747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:55.384575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:25:55.384575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:25:55.384575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:55.384575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:55.384575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 247/515] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:55.384829 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d216a02-36e3-485c-af9c-baad0b4cf825 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:55.410830 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132346c-187c-4c3a-b0a9-a78d8d4a21af None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:55.443934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a2d98-001a-4225-9a2a-f5b2847dad26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:55.444571 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a2d98-001a-4225-9a2a-f5b2847dad26 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:55.444855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a2d98-001a-4225-9a2a-f5b2847dad26 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:55.445142 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a2d98-001a-4225-9a2a-f5b2847dad26 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:55.445976 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a2d98-001a-4225-9a2a-f5b2847dad26 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:55.446287 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a2d98-001a-4225-9a2a-f5b2847dad26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:55.447771 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a2d98-001a-4225-9a2a-f5b2847dad26 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:55.465288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 269/516] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Jun 23 11:25:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:55.466075 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a2d98-001a-4225-9a2a-f5b2847dad26 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:55.483907 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d216a02-36e3-485c-af9c-baad0b4cf825 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:55.501326 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:55.511918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:55.520029 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f5696-ea59-48c5-aa4a-e3504a626410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:55.520687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f5696-ea59-48c5-aa4a-e3504a626410 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:55.520966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f5696-ea59-48c5-aa4a-e3504a626410 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:55.521232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f5696-ea59-48c5-aa4a-e3504a626410 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:55.522124 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f5696-ea59-48c5-aa4a-e3504a626410 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:55.522429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f5696-ea59-48c5-aa4a-e3504a626410 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:55.523847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f5696-ea59-48c5-aa4a-e3504a626410 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:55.535699 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 248/517] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:55.536492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f5696-ea59-48c5-aa4a-e3504a626410 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:55.571468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a2d98-001a-4225-9a2a-f5b2847dad26 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:55.605475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd178b-a4c5-4cc9-a579-be63b5832ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:55.606266 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd178b-a4c5-4cc9-a579-be63b5832ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:55.606560 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd178b-a4c5-4cc9-a579-be63b5832ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:55.606843 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd178b-a4c5-4cc9-a579-be63b5832ab4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:55.616990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd178b-a4c5-4cc9-a579-be63b5832ab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:55.618447 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd178b-a4c5-4cc9-a579-be63b5832ab4 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:55.634769 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f5696-ea59-48c5-aa4a-e3504a626410 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:55.636552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-05fd178b-a4c5-4cc9-a579-be63b5832ab4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:55.636552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:55.636552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:55.636552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:55.636552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:55.636552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:55.636552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:55.636552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:55.636552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:55.636552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:55.636552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:55.636552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:55.636552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:25:55.636552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:25:55.636552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:25:55.637346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:25:55.637346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:55.637346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:55.637346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:55.637346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:55.637346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:55.637346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:55.637346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:55.637346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:55.637346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:55.637346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:55.637346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:55.637346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:55.637346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:55.637346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:55.637346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:55.637346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:55.638347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:25:55.638347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:25:55.638347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:55.638347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:55.638347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 270/518] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:55.639446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd178b-a4c5-4cc9-a579-be63b5832ab4 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:55.669226 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15390fd6-4cec-4218-9611-8b705af91be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:55.669925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15390fd6-4cec-4218-9611-8b705af91be7 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:55.670198 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15390fd6-4cec-4218-9611-8b705af91be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:55.670471 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15390fd6-4cec-4218-9611-8b705af91be7 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:55.671199 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15390fd6-4cec-4218-9611-8b705af91be7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:55.672612 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15390fd6-4cec-4218-9611-8b705af91be7 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:55.721896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:55.735890 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd178b-a4c5-4cc9-a579-be63b5832ab4 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:55.741036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 249/519] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 23 11:25:55 2022] POST /identity/v3/users => generated 247 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:25:55.741840 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15390fd6-4cec-4218-9611-8b705af91be7 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:55.773412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf952d-1d15-4e83-9434-092c43f357eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:55.774090 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf952d-1d15-4e83-9434-092c43f357eb None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:55.774375 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf952d-1d15-4e83-9434-092c43f357eb None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:55.774678 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf952d-1d15-4e83-9434-092c43f357eb None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:55.775418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf952d-1d15-4e83-9434-092c43f357eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:55.775725 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf952d-1d15-4e83-9434-092c43f357eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:55.777151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf952d-1d15-4e83-9434-092c43f357eb None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:55.790630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 271/520] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:55.791637 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf952d-1d15-4e83-9434-092c43f357eb None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:55.815599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:55.877408 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15390fd6-4cec-4218-9611-8b705af91be7 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:55.911483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf952d-1d15-4e83-9434-092c43f357eb None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:55.916635 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfa790-25f3-4433-991d-e3cdcf865168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:55.917522 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfa790-25f3-4433-991d-e3cdcf865168 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:55.917626 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfa790-25f3-4433-991d-e3cdcf865168 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:55.917830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfa790-25f3-4433-991d-e3cdcf865168 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:55.956264 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05611e91-8b9a-4756-9d69-fc7504cf0cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:55.957417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05611e91-8b9a-4756-9d69-fc7504cf0cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:55.957739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05611e91-8b9a-4756-9d69-fc7504cf0cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:55.958019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05611e91-8b9a-4756-9d69-fc7504cf0cc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:55.965652 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05611e91-8b9a-4756-9d69-fc7504cf0cc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:55.967942 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05611e91-8b9a-4756-9d69-fc7504cf0cc3 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:55.983122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-05611e91-8b9a-4756-9d69-fc7504cf0cc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:25:55.983122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:55.983122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:55.983122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:55.983122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:55.983122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:55.983122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:55.983122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:55.983122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:55.983122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:55.983122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:55.983122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:55.983122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:25:55.983122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:25:55.983122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:25:55.984002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:25:55.984002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:55.984002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:55.984002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:55.984002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:55.984002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:55.984002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:55.984002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:55.984002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:55.984002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:55.984002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:55.984002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:55.984002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:55.984002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:55.984002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:55.984002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:55.984002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:55.985132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:25:55.985132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:25:55.985132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:25:55.985132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:25:55.985132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:25:55.985132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:25:55.985132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:25:55.985132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:55.985132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 272/521] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:25:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:55.985132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05611e91-8b9a-4756-9d69-fc7504cf0cc3 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:55.995595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:56.000928 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cfa790-25f3-4433-991d-e3cdcf865168 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:56.002390 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cfa790-25f3-4433-991d-e3cdcf865168 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:56.053249 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:56.092151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05611e91-8b9a-4756-9d69-fc7504cf0cc3 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:56.117315 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 250/522] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:56.117315 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfa790-25f3-4433-991d-e3cdcf865168 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:56.133397 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79626e06-f7d5-41ce-824b-1402be50a3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:56.134065 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79626e06-f7d5-41ce-824b-1402be50a3f3 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:56.134349 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79626e06-f7d5-41ce-824b-1402be50a3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:56.134654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79626e06-f7d5-41ce-824b-1402be50a3f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:56.135393 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79626e06-f7d5-41ce-824b-1402be50a3f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:56.135710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79626e06-f7d5-41ce-824b-1402be50a3f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:56.137089 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79626e06-f7d5-41ce-824b-1402be50a3f3 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:56.149815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 273/523] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:56.152326 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79626e06-f7d5-41ce-824b-1402be50a3f3 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:56.221401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfa790-25f3-4433-991d-e3cdcf865168 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:56.222122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:56.257562 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3b480-7ffe-4112-93ce-c4b13ba8fecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:56.258361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3b480-7ffe-4112-93ce-c4b13ba8fecf None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:56.258719 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3b480-7ffe-4112-93ce-c4b13ba8fecf None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:56.259009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3b480-7ffe-4112-93ce-c4b13ba8fecf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:56.261052 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79626e06-f7d5-41ce-824b-1402be50a3f3 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:56.267135 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3b480-7ffe-4112-93ce-c4b13ba8fecf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:56.268578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3b480-7ffe-4112-93ce-c4b13ba8fecf None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:56.284074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-a2e3b480-7ffe-4112-93ce-c4b13ba8fecf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 11:25:56.284074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:56.284074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:56.284074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:56.284074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:56.284074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:56.284074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:56.284074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:56.284074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:56.284074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:56.284074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:56.284074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:56.284074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:25:56.284074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:25:56.284074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:25:56.284074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:25:56.285021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:56.285021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:56.285021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:25:56.285021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:25:56.285021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:25:56.285021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:25:56.285021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:56.285021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:56.285021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:56.285021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:56.285021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:56.285021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:56.285021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:56.285021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:56.285021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:56.285021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:56.285021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:56.285021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:56.285997 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:56.285997 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:56.285997 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:25:56.285997 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:25:56.285997 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:25:56.285997 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:25:56.285997 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:25:56.285997 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:25:56.285997 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 11:25:56.285997 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:56.285997 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 251/524] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:25:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:56.286584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3b480-7ffe-4112-93ce-c4b13ba8fecf None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:56.291052 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:56.304414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec83f3a-7e76-422d-b3e0-3474a32857d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:56.306386 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec83f3a-7e76-422d-b3e0-3474a32857d0 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:56.306386 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec83f3a-7e76-422d-b3e0-3474a32857d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:56.306386 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec83f3a-7e76-422d-b3e0-3474a32857d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:56.306629 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec83f3a-7e76-422d-b3e0-3474a32857d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:56.306895 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec83f3a-7e76-422d-b3e0-3474a32857d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:56.308337 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec83f3a-7e76-422d-b3e0-3474a32857d0 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:56.328985 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 274/525] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:25:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:56.328985 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec83f3a-7e76-422d-b3e0-3474a32857d0 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:56.395087 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3b480-7ffe-4112-93ce-c4b13ba8fecf None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:56.430850 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae78bfa-fe07-4035-86f1-64e86303e9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:56.431705 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae78bfa-fe07-4035-86f1-64e86303e9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:56.431985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae78bfa-fe07-4035-86f1-64e86303e9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:56.432313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae78bfa-fe07-4035-86f1-64e86303e9e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:56.439951 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec83f3a-7e76-422d-b3e0-3474a32857d0 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:56.447345 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae78bfa-fe07-4035-86f1-64e86303e9e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:56.448835 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae78bfa-fe07-4035-86f1-64e86303e9e3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:56.467021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-0ae78bfa-fe07-4035-86f1-64e86303e9e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:56.467021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:56.467021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:56.467021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:56.467021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:56.467021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:56.467021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:56.467021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:56.467021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:56.467021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:56.467021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:56.467021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:56.467021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:25:56.467021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:25:56.467021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:25:56.467896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:25:56.467896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:56.467896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:56.467896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:56.467896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:56.467896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:56.467896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:56.467896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:56.467896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:56.467896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:56.467896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:56.467896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:56.467896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:56.467896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:56.467896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:56.467896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:56.467896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:56.468813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:25:56.468813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:25:56.468813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:56.468813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:56.468813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 252/526] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:56.469085 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae78bfa-fe07-4035-86f1-64e86303e9e3 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:56.469997 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:56.481372 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714d617-f841-489d-bcb0-ee9d416fd8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:56.485464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714d617-f841-489d-bcb0-ee9d416fd8fd None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:56.485757 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714d617-f841-489d-bcb0-ee9d416fd8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:56.486033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714d617-f841-489d-bcb0-ee9d416fd8fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:56.496351 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e714d617-f841-489d-bcb0-ee9d416fd8fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:56.498384 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e714d617-f841-489d-bcb0-ee9d416fd8fd None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:56.512340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-e714d617-f841-489d-bcb0-ee9d416fd8fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:25:56.512340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:56.512340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:56.512340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:56.512340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:56.512340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:56.512340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:56.512340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:56.512340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:56.512340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:56.512340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:56.512340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:56.512340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:25:56.512340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:25:56.512340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:25:56.513370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:25:56.513370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:56.513370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:56.513370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:56.513370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:56.513370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:56.513370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:56.513370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:56.513370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:56.513370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:56.513370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:56.513370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:56.513370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:56.513370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:56.513370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:56.513370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:56.513370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:56.515536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:25:56.515536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:25:56.515536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:25:56.515536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:25:56.515536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:25:56.515536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:25:56.515536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:25:56.515536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:56.515536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 275/527] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:25:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:56.515536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714d617-f841-489d-bcb0-ee9d416fd8fd None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:56.538674 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:56.586707 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae78bfa-fe07-4035-86f1-64e86303e9e3 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:56.616782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714d617-f841-489d-bcb0-ee9d416fd8fd None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:56.620724 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a5487-770e-4f10-b700-4795f8587c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:56.621400 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a5487-770e-4f10-b700-4795f8587c15 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:56.621685 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a5487-770e-4f10-b700-4795f8587c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:56.621963 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a5487-770e-4f10-b700-4795f8587c15 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:56.623454 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a5487-770e-4f10-b700-4795f8587c15 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:56.623775 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a5487-770e-4f10-b700-4795f8587c15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:56.625220 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a5487-770e-4f10-b700-4795f8587c15 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:56.650715 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79461df0-396e-4ff1-89ac-bc48041b73a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:56.651367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79461df0-396e-4ff1-89ac-bc48041b73a4 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:56.651649 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79461df0-396e-4ff1-89ac-bc48041b73a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:56.651921 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79461df0-396e-4ff1-89ac-bc48041b73a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:56.652742 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79461df0-396e-4ff1-89ac-bc48041b73a4 None admin] RBAC: Adding query filter params (role.id=13708a4819a94b94b47ba5994aaa9310, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=161c5ed1e47e450290a1f428ee100b68) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:56.653053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79461df0-396e-4ff1-89ac-bc48041b73a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:56.654695 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79461df0-396e-4ff1-89ac-bc48041b73a4 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:56.665849 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 276/528] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:25:56 2022] GET /identity/v3/role_assignments?user.id=161c5ed1e47e450290a1f428ee100b68&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=13708a4819a94b94b47ba5994aaa9310 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:56.665849 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79461df0-396e-4ff1-89ac-bc48041b73a4 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:56.666092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 253/529] 10.222.0.13 () {56 vars in 1139 bytes} [Thu Jun 23 11:25:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:56.666092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a5487-770e-4f10-b700-4795f8587c15 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:56.729798 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:56.787062 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79461df0-396e-4ff1-89ac-bc48041b73a4 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:56.811310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a5487-770e-4f10-b700-4795f8587c15 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:56.831479 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:56.837352 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74330613-f56e-4e06-8fda-04040e064fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:56.838028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74330613-f56e-4e06-8fda-04040e064fdc None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:56.838822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74330613-f56e-4e06-8fda-04040e064fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:56.839111 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74330613-f56e-4e06-8fda-04040e064fdc None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:56.839858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74330613-f56e-4e06-8fda-04040e064fdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:56.840173 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74330613-f56e-4e06-8fda-04040e064fdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:56.841590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74330613-f56e-4e06-8fda-04040e064fdc None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:56.854565 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 277/530] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:56.855709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc4762-369e-45b8-a635-d65fbc00b77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:56.855824 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74330613-f56e-4e06-8fda-04040e064fdc None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:56.855898 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc4762-369e-45b8-a635-d65fbc00b77b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:56.855898 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc4762-369e-45b8-a635-d65fbc00b77b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:56.855898 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc4762-369e-45b8-a635-d65fbc00b77b None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:56.856576 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc4762-369e-45b8-a635-d65fbc00b77b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:56.859594 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc4762-369e-45b8-a635-d65fbc00b77b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:56.859594 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc4762-369e-45b8-a635-d65fbc00b77b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:56.878902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 254/531] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:25:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:56.879749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc4762-369e-45b8-a635-d65fbc00b77b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:56.978933 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74330613-f56e-4e06-8fda-04040e064fdc None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:56.994563 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:57.005569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc4762-369e-45b8-a635-d65fbc00b77b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:57.026194 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90305448-5707-41e1-b7a2-ae2fb0737c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:57.027479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90305448-5707-41e1-b7a2-ae2fb0737c30 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:57.027972 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90305448-5707-41e1-b7a2-ae2fb0737c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:57.028433 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90305448-5707-41e1-b7a2-ae2fb0737c30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:57.043337 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c127acb-ba7a-420c-9ab5-abeec64050d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:57.044056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c127acb-ba7a-420c-9ab5-abeec64050d6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:57.045013 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c127acb-ba7a-420c-9ab5-abeec64050d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:57.045449 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c127acb-ba7a-420c-9ab5-abeec64050d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:57.053141 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c127acb-ba7a-420c-9ab5-abeec64050d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:57.054722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c127acb-ba7a-420c-9ab5-abeec64050d6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:57.071876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-6c127acb-ba7a-420c-9ab5-abeec64050d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:25:57.071876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:57.071876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:57.071876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:57.071876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:57.071876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:57.071876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:57.071876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:57.071876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:57.071876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:57.071876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:57.071876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:57.071876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:25:57.071876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:25:57.071876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:25:57.072746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:25:57.072746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:57.072746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:57.072746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:57.072746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:57.072746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:57.072746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:57.072746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:57.072746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:57.072746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:57.072746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:57.072746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:57.072746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:57.072746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:57.072746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:57.072746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:57.072746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:57.073994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:25:57.073994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:25:57.073994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:25:57.073994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:25:57.073994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:25:57.073994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:25:57.073994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:25:57.073994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:57.073994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 255/532] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:25:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:57.074463 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c127acb-ba7a-420c-9ab5-abeec64050d6 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:57.115250 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90305448-5707-41e1-b7a2-ae2fb0737c30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:57.117520 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90305448-5707-41e1-b7a2-ae2fb0737c30 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:57.126190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:57.202812 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c127acb-ba7a-420c-9ab5-abeec64050d6 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:57.244419 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 278/533] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:25:57.245400 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90305448-5707-41e1-b7a2-ae2fb0737c30 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:57.251011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c7fe3-dce2-4af5-90ed-e94277992ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:57.251965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c7fe3-dce2-4af5-90ed-e94277992ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:57.252502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c7fe3-dce2-4af5-90ed-e94277992ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:57.252980 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c7fe3-dce2-4af5-90ed-e94277992ef5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:57.254564 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c7fe3-dce2-4af5-90ed-e94277992ef5 None admin] RBAC: Adding query filter params (role.id=13708a4819a94b94b47ba5994aaa9310, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=b6105838cbf1409587ee7d9e3be554f8) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:57.255090 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c7fe3-dce2-4af5-90ed-e94277992ef5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:57.256733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c7fe3-dce2-4af5-90ed-e94277992ef5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:57.262491 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:57.265993 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 256/534] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:25:57 2022] GET /identity/v3/role_assignments?user.id=b6105838cbf1409587ee7d9e3be554f8&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=13708a4819a94b94b47ba5994aaa9310 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:57.267127 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c7fe3-dce2-4af5-90ed-e94277992ef5 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:57.390286 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:57.400727 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c7fe3-dce2-4af5-90ed-e94277992ef5 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:57.464560 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90305448-5707-41e1-b7a2-ae2fb0737c30 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:57.468559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da52764-661b-49c1-9734-995f1378b29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:57.469377 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da52764-661b-49c1-9734-995f1378b29f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:57.469658 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da52764-661b-49c1-9734-995f1378b29f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:57.469930 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da52764-661b-49c1-9734-995f1378b29f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:57.499840 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da52764-661b-49c1-9734-995f1378b29f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:57.501310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da52764-661b-49c1-9734-995f1378b29f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:57.531267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-2da52764-661b-49c1-9734-995f1378b29f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:57.531267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:57.531267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:57.531267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:57.531267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:57.531267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:57.531267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:57.531267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:57.531267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:57.531267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:57.531267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:57.531267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:57.531267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:25:57.531267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:25:57.531267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:25:57.532164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:25:57.532164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:57.532164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:57.532164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:57.532164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:57.532164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:57.532164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:57.532164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:57.532164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:57.532164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:57.532164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:57.532164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:57.532164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:57.532164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:57.532164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:57.532164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:57.532164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:57.533690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:25:57.533690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:25:57.533690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:57.533690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:57.533905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c82510-4b0f-4286-8a9f-09f4f39e3907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:57.533905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c82510-4b0f-4286-8a9f-09f4f39e3907 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:57.533905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c82510-4b0f-4286-8a9f-09f4f39e3907 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:57.533905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c82510-4b0f-4286-8a9f-09f4f39e3907 None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:57.534171 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 257/535] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:57.534235 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c82510-4b0f-4286-8a9f-09f4f39e3907 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:57.534235 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c82510-4b0f-4286-8a9f-09f4f39e3907 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:57.536484 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c82510-4b0f-4286-8a9f-09f4f39e3907 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:57.537916 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da52764-661b-49c1-9734-995f1378b29f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:57.553168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 279/536] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:25:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:57.558675 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c82510-4b0f-4286-8a9f-09f4f39e3907 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:57.698972 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:57.707088 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da52764-661b-49c1-9734-995f1378b29f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:57.751107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838df73c-63c7-43c8-a15a-8da17521ff21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:57.751774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838df73c-63c7-43c8-a15a-8da17521ff21 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:57.752055 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838df73c-63c7-43c8-a15a-8da17521ff21 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:57.752325 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838df73c-63c7-43c8-a15a-8da17521ff21 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:57.753062 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838df73c-63c7-43c8-a15a-8da17521ff21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:57.753366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838df73c-63c7-43c8-a15a-8da17521ff21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:57.754773 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838df73c-63c7-43c8-a15a-8da17521ff21 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:57.773230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 258/537] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:57.779018 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838df73c-63c7-43c8-a15a-8da17521ff21 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:57.783862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c82510-4b0f-4286-8a9f-09f4f39e3907 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:57.793262 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:57.853435 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8002c-2289-4a8b-90bc-89f3e29f7237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:57.854380 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8002c-2289-4a8b-90bc-89f3e29f7237 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:57.854894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8002c-2289-4a8b-90bc-89f3e29f7237 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:57.855317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8002c-2289-4a8b-90bc-89f3e29f7237 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:57.856324 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8002c-2289-4a8b-90bc-89f3e29f7237 None admin] RBAC: Adding query filter params (role.id=13708a4819a94b94b47ba5994aaa9310, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=871dd684032c49d09608f0af05b7a765) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:57.856786 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8002c-2289-4a8b-90bc-89f3e29f7237 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:57.859108 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8002c-2289-4a8b-90bc-89f3e29f7237 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:57.872150 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 280/538] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:25:57 2022] GET /identity/v3/role_assignments?user.id=871dd684032c49d09608f0af05b7a765&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=13708a4819a94b94b47ba5994aaa9310 => generated 259 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:57.895571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838df73c-63c7-43c8-a15a-8da17521ff21 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:57.932852 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4170c-582f-4e7c-b21c-b3ebd02c67a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:57.933648 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4170c-582f-4e7c-b21c-b3ebd02c67a1 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:57.933928 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4170c-582f-4e7c-b21c-b3ebd02c67a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:57.934201 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4170c-582f-4e7c-b21c-b3ebd02c67a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:57.943022 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4170c-582f-4e7c-b21c-b3ebd02c67a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:57.944507 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4170c-582f-4e7c-b21c-b3ebd02c67a1 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:57.960695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-04b4170c-582f-4e7c-b21c-b3ebd02c67a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:25:57.960695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:57.960695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:57.960695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:57.960695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:57.960695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:57.960695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:57.960695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:57.960695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:57.960695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:57.960695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:57.960695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:57.960695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:25:57.960695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:25:57.960695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:25:57.961575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:25:57.961575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:57.961575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:57.961575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:57.961575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:57.961575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:57.961575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:57.961575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:57.961575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:57.961575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:57.961575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:57.961575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:57.961575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:57.961575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:57.961575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:57.961575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:57.961575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:57.962433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:25:57.962433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:25:57.962433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:25:57.962433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:25:57.962433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:25:57.962433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:25:57.962433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:25:57.962433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:57.962433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 259/539] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:25:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:57.968700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8002c-2289-4a8b-90bc-89f3e29f7237 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:57.994883 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2d083-f72f-4f59-8ad0-7f8a128c562f None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:57.995197 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2d083-f72f-4f59-8ad0-7f8a128c562f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:57.995439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2d083-f72f-4f59-8ad0-7f8a128c562f None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:57.997121 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 260/540] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:58.002946 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b91b7-79ea-4fe0-82f3-8c6bc56d8fe3 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:58.003272 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b91b7-79ea-4fe0-82f3-8c6bc56d8fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:58.003505 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b91b7-79ea-4fe0-82f3-8c6bc56d8fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:58.031564 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-312b91b7-79ea-4fe0-82f3-8c6bc56d8fe3 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:58.032773 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-312b91b7-79ea-4fe0-82f3-8c6bc56d8fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:58.140647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8002c-2289-4a8b-90bc-89f3e29f7237 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:58.151526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:58.160369 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 261/541] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:25:58.195305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c25bb-3cc6-4a36-99ee-88bc2cba8cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:58.195972 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c25bb-3cc6-4a36-99ee-88bc2cba8cee None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:58.196252 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c25bb-3cc6-4a36-99ee-88bc2cba8cee None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:58.196527 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c25bb-3cc6-4a36-99ee-88bc2cba8cee None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:58.197311 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c25bb-3cc6-4a36-99ee-88bc2cba8cee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:58.197617 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c25bb-3cc6-4a36-99ee-88bc2cba8cee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:58.199225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c25bb-3cc6-4a36-99ee-88bc2cba8cee None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:58.201282 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:58.217123 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 281/542] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:25:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:58.223036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b91b7-79ea-4fe0-82f3-8c6bc56d8fe3 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:58.237120 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c25bb-3cc6-4a36-99ee-88bc2cba8cee None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:58.394691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c25bb-3cc6-4a36-99ee-88bc2cba8cee None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:58.429922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b91b7-79ea-4fe0-82f3-8c6bc56d8fe3 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:58.458947 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da65bf-9d67-4992-92e9-3b3cf41c6fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:58.460109 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da65bf-9d67-4992-92e9-3b3cf41c6fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:58.460597 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da65bf-9d67-4992-92e9-3b3cf41c6fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:58.461089 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da65bf-9d67-4992-92e9-3b3cf41c6fb9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:58.480502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da65bf-9d67-4992-92e9-3b3cf41c6fb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:58.489634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-03da65bf-9d67-4992-92e9-3b3cf41c6fb9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:25:58.489634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:58.489634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:58.489634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:58.489634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:58.489634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:58.489634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:58.489634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:58.489634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:58.489634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:58.489634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:58.489634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:58.489634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:58.489634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:58.489634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:58.489634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:58.490550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:58.490550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:58.490550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:58.490550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:58.490550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:58.490550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:58.490550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:58.490550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:58.490550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:58.490550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:58.490550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:58.490550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:58.490550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:58.490550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:58.490550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:58.490550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:58.491366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:58.491366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:58.491366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:58.491366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:58.491366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:58.491366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:58.491366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:58.491366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:58.491366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:58.491366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:58.491366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:25:58.491366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:58.494063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 282/543] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:25:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:58.501669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da65bf-9d67-4992-92e9-3b3cf41c6fb9 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:58.502059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ba187-af9d-4fe0-a3ae-79e8a1089428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:58.502780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ba187-af9d-4fe0-a3ae-79e8a1089428 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:58.503065 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ba187-af9d-4fe0-a3ae-79e8a1089428 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:58.503339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ba187-af9d-4fe0-a3ae-79e8a1089428 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:58.504192 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583ba187-af9d-4fe0-a3ae-79e8a1089428 None admin] RBAC: Adding query filter params (role.id=13708a4819a94b94b47ba5994aaa9310, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=810fa3f54cd04f989252765f7e76e602) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:58.504504 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583ba187-af9d-4fe0-a3ae-79e8a1089428 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:58.505955 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583ba187-af9d-4fe0-a3ae-79e8a1089428 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:58.512373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 262/544] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:25:58 2022] GET /identity/v3/role_assignments?user.id=810fa3f54cd04f989252765f7e76e602&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=13708a4819a94b94b47ba5994aaa9310 => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:58.633218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:58.687711 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da65bf-9d67-4992-92e9-3b3cf41c6fb9 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:58.724876 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad69877-938e-4a73-9b02-4ba44f2ca8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:58.725634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad69877-938e-4a73-9b02-4ba44f2ca8b1 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:58.725831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad69877-938e-4a73-9b02-4ba44f2ca8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:58.726111 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad69877-938e-4a73-9b02-4ba44f2ca8b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:58.726879 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad69877-938e-4a73-9b02-4ba44f2ca8b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:58.727192 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad69877-938e-4a73-9b02-4ba44f2ca8b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:58.728555 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad69877-938e-4a73-9b02-4ba44f2ca8b1 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:58.738639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 283/545] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:25:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:25:58.748179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ba187-af9d-4fe0-a3ae-79e8a1089428 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:58.867537 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:58.907935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ba187-af9d-4fe0-a3ae-79e8a1089428 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:58.928300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e724bca-6875-4ab6-a98c-f6d7e2f60c60 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:58.929387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e724bca-6875-4ab6-a98c-f6d7e2f60c60 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:58.929666 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e724bca-6875-4ab6-a98c-f6d7e2f60c60 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:58.935342 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 284/546] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:58.937007 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4fa98-8030-4c99-9912-5c716e2271ce None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:58.937298 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4fa98-8030-4c99-9912-5c716e2271ce None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:58.937574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4fa98-8030-4c99-9912-5c716e2271ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:58.955722 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-e6c4fa98-8030-4c99-9912-5c716e2271ce None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:25:58.956911 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e6c4fa98-8030-4c99-9912-5c716e2271ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:58.971793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eeb2aa-1958-4846-bdc3-0eb3363789aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:58.972603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eeb2aa-1958-4846-bdc3-0eb3363789aa None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:58.972891 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eeb2aa-1958-4846-bdc3-0eb3363789aa None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:58.973167 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eeb2aa-1958-4846-bdc3-0eb3363789aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:58.993356 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eeb2aa-1958-4846-bdc3-0eb3363789aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:58.994825 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eeb2aa-1958-4846-bdc3-0eb3363789aa None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:59.029654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-50eeb2aa-1958-4846-bdc3-0eb3363789aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:59.029654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:59.029654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:59.029654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:59.029654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:59.029654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:59.029654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:59.029654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:59.029654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:59.029654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:59.029654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:59.029654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:59.029654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:25:59.029654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:25:59.029654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:25:59.031774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:25:59.031774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:59.031774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:59.031774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:59.031774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:59.031774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:59.031774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:59.031774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:59.031774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:59.031774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:59.031774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:59.031774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:59.031774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:59.031774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:59.031774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:59.031774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:59.031774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:59.033791 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:25:59.033791 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:25:59.033791 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:59.033791 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:59.033791 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 263/547] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:59.037218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eeb2aa-1958-4846-bdc3-0eb3363789aa None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:59.072544 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 285/548] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:25:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:25:59.095073 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4fa98-8030-4c99-9912-5c716e2271ce None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:59.161248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:59.200687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eeb2aa-1958-4846-bdc3-0eb3363789aa None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:59.218052 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:59.237736 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00600b01-0a48-4777-95de-b6d862e555aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:59.238416 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00600b01-0a48-4777-95de-b6d862e555aa None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:59.238810 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00600b01-0a48-4777-95de-b6d862e555aa None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:59.239090 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00600b01-0a48-4777-95de-b6d862e555aa None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:59.239826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00600b01-0a48-4777-95de-b6d862e555aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:59.240151 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00600b01-0a48-4777-95de-b6d862e555aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:59.241520 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00600b01-0a48-4777-95de-b6d862e555aa None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:59.256081 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 264/549] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:59.264073 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00600b01-0a48-4777-95de-b6d862e555aa None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:59.291772 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4fa98-8030-4c99-9912-5c716e2271ce None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:59.360618 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c4627-f561-4a77-a80c-6fa488b33144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:59.361421 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c4627-f561-4a77-a80c-6fa488b33144 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:59.361705 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c4627-f561-4a77-a80c-6fa488b33144 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:59.361972 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c4627-f561-4a77-a80c-6fa488b33144 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:59.383359 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c4627-f561-4a77-a80c-6fa488b33144 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:25:59.392061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-2e1c4627-f561-4a77-a80c-6fa488b33144 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:25:59.392061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:59.392061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:59.392061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:59.392061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:59.392061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:59.392061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:59.392061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:59.392061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:59.392061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:59.392061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:59.392061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:59.392061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:25:59.392061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:25:59.392061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:25:59.392061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:25:59.392964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:25:59.392964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:25:59.392964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:25:59.392964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:25:59.392964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:59.392964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:59.392964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:59.392964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:59.392964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:59.392964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:59.392964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:59.392964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:59.392964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:59.392964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:59.392964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:59.392964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:59.393781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:59.393781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:59.393781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:59.393781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:59.393781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:25:59.393781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:25:59.393781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:25:59.393781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:25:59.393781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:25:59.393781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:25:59.393781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:25:59.393781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:59.393781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 286/550] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:25:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:25:59.399640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c4627-f561-4a77-a80c-6fa488b33144 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:59.441883 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00600b01-0a48-4777-95de-b6d862e555aa None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:59.502067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30870a-b47d-4359-b137-e032b2b9d2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:59.502895 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30870a-b47d-4359-b137-e032b2b9d2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:59.503180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30870a-b47d-4359-b137-e032b2b9d2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:59.503454 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30870a-b47d-4359-b137-e032b2b9d2a1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:59.516574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d30870a-b47d-4359-b137-e032b2b9d2a1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:59.517963 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d30870a-b47d-4359-b137-e032b2b9d2a1 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:59.546312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-1d30870a-b47d-4359-b137-e032b2b9d2a1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:25:59.546312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:59.546312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:59.546312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:59.546312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:59.546312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:59.546312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:59.546312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:59.546312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:59.546312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:59.546312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:59.546312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:59.546312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:25:59.546312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:25:59.546312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:25:59.546312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:25:59.547448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:59.547448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:59.547448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:25:59.547448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:25:59.547448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:25:59.547448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:25:59.547448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:59.547448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:59.547448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:59.547448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:59.547448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:59.547448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:59.547448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:59.547448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:59.547448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:59.547448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:59.547448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:59.547448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:59.548341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:59.548341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:59.548341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:25:59.548341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:25:59.548341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:25:59.548341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:25:59.548341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:25:59.548341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:25:59.548341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:25:59.548341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:25:59.548341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 265/551] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:25:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:25:59.552998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30870a-b47d-4359-b137-e032b2b9d2a1 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:59.594782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c4627-f561-4a77-a80c-6fa488b33144 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:59.606920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:59.630891 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:59.659157 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cb3db-3850-4c4c-adbf-18f6d7242e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:59.660540 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cb3db-3850-4c4c-adbf-18f6d7242e38 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:59.661032 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cb3db-3850-4c4c-adbf-18f6d7242e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:59.661498 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cb3db-3850-4c4c-adbf-18f6d7242e38 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:59.662461 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9cb3db-3850-4c4c-adbf-18f6d7242e38 None admin] RBAC: Adding query filter params (name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:59.666989 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9cb3db-3850-4c4c-adbf-18f6d7242e38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:59.668428 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9cb3db-3850-4c4c-adbf-18f6d7242e38 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:59.682119 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 287/552] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:25:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:59.687592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cb3db-3850-4c4c-adbf-18f6d7242e38 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:59.706111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30870a-b47d-4359-b137-e032b2b9d2a1 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:59.741101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75575422-c166-4bcc-bbad-cf2eac3bb4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:59.741774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75575422-c166-4bcc-bbad-cf2eac3bb4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:59.742361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75575422-c166-4bcc-bbad-cf2eac3bb4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:59.743394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75575422-c166-4bcc-bbad-cf2eac3bb4e3 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:59.744474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75575422-c166-4bcc-bbad-cf2eac3bb4e3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:25:59.744928 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75575422-c166-4bcc-bbad-cf2eac3bb4e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:59.746507 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75575422-c166-4bcc-bbad-cf2eac3bb4e3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:59.765616 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 266/553] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Jun 23 11:25:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:25:59.773783 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75575422-c166-4bcc-bbad-cf2eac3bb4e3 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:59.880799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cb3db-3850-4c4c-adbf-18f6d7242e38 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:59.908225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:59.915001 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75575422-c166-4bcc-bbad-cf2eac3bb4e3 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:59.932379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8c1ac-4184-40d0-affb-6fe6b304b26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:59.933430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8c1ac-4184-40d0-affb-6fe6b304b26b None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:59.933719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8c1ac-4184-40d0-affb-6fe6b304b26b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:59.934045 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8c1ac-4184-40d0-affb-6fe6b304b26b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:59.949888 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8c1ac-4184-40d0-affb-6fe6b304b26b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:59.951499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2771d531-5fd6-428e-83db-6417205fa2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:59.952059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8c1ac-4184-40d0-affb-6fe6b304b26b None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:59.952736 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771d531-5fd6-428e-83db-6417205fa2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:59.953192 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771d531-5fd6-428e-83db-6417205fa2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:59.953649 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771d531-5fd6-428e-83db-6417205fa2f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:59.973541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-74f8c1ac-4184-40d0-affb-6fe6b304b26b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:59.973541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:25:59.973541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:25:59.973541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:25:59.973541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:25:59.973541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:25:59.973541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:25:59.973541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:25:59.973541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:25:59.973541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:25:59.973541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:25:59.973541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:25:59.973541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:25:59.973541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:25:59.973541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:25:59.974450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:25:59.974450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:25:59.974450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:25:59.974450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:25:59.974450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:25:59.974450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:25:59.974450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:25:59.974450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:25:59.974450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:25:59.974450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:25:59.974450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:25:59.974450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:25:59.974450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:25:59.974450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:25:59.974450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:25:59.974450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:25:59.974450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:25:59.975334 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:25:59.975334 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:25:59.975334 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:25:59.975334 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:25:59.977528 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 288/554] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:25:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:25:59.979029 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddf6f9-3818-4cc8-9e6d-f016d99185b5 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:59.979551 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddf6f9-3818-4cc8-9e6d-f016d99185b5 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:59.980032 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddf6f9-3818-4cc8-9e6d-f016d99185b5 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:59.981870 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 289/555] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:25:59.983277 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddf6f9-3818-4cc8-9e6d-f016d99185b5 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:00.009816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:00.056077 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2771d531-5fd6-428e-83db-6417205fa2f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:00.057982 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2771d531-5fd6-428e-83db-6417205fa2f2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:00.105967 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddf6f9-3818-4cc8-9e6d-f016d99185b5 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:00.144145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e85423-7ea8-45de-868d-1734315a7f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:00.145148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e85423-7ea8-45de-868d-1734315a7f41 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:00.145627 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e85423-7ea8-45de-868d-1734315a7f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:00.146094 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e85423-7ea8-45de-868d-1734315a7f41 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:00.147116 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e85423-7ea8-45de-868d-1734315a7f41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:00.147643 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e85423-7ea8-45de-868d-1734315a7f41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:00.149202 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e85423-7ea8-45de-868d-1734315a7f41 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:00.161804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 290/556] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:00.163386 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51c35f-5846-4289-a420-167d1cc74156 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:00.163913 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51c35f-5846-4289-a420-167d1cc74156 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:00.164394 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51c35f-5846-4289-a420-167d1cc74156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:00.170696 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:00.184732 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 267/557] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:25:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:26:00.184732 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2771d531-5fd6-428e-83db-6417205fa2f2 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:00.187369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-de51c35f-5846-4289-a420-167d1cc74156 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:00.188762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-de51c35f-5846-4289-a420-167d1cc74156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:00.260162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 291/558] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:26:00.261406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51c35f-5846-4289-a420-167d1cc74156 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:00.274129 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:00.322669 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2771d531-5fd6-428e-83db-6417205fa2f2 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:00.366956 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c930e-c412-4e7f-8c68-08498d31171c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:00.367758 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51c35f-5846-4289-a420-167d1cc74156 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:00.367868 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c930e-c412-4e7f-8c68-08498d31171c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:00.368040 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c930e-c412-4e7f-8c68-08498d31171c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:00.368312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c930e-c412-4e7f-8c68-08498d31171c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:00.378425 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c930e-c412-4e7f-8c68-08498d31171c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:00.379939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c930e-c412-4e7f-8c68-08498d31171c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:00.396308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-637c930e-c412-4e7f-8c68-08498d31171c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:26:00.396308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:00.396308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:00.396308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:00.396308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:00.396308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:00.396308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:00.396308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:00.396308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:00.396308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:00.396308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:00.396308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:00.396308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:00.396308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:00.396308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:00.396308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:00.397382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:00.397382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:00.397382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:00.397382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:00.397382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:00.397382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:00.397382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:00.397382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:00.397382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:00.397382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:00.397382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:00.397382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:00.397382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:00.397382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:00.397382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:00.397382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:00.397382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:00.397382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:00.398295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:00.398295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:00.398295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:00.398295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:00.398295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:00.398295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:00.398295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:00.398295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:00.398295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:26:00.398295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:00.398295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 268/559] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jun 23 11:26:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:00.398295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c930e-c412-4e7f-8c68-08498d31171c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:00.413546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f85cf8-a3a6-4bf9-b55f-4c26c1b39832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:00.414342 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f85cf8-a3a6-4bf9-b55f-4c26c1b39832 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:00.414647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f85cf8-a3a6-4bf9-b55f-4c26c1b39832 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:00.414922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f85cf8-a3a6-4bf9-b55f-4c26c1b39832 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:00.426069 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f85cf8-a3a6-4bf9-b55f-4c26c1b39832 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:00.427620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f85cf8-a3a6-4bf9-b55f-4c26c1b39832 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:00.446567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-77f85cf8-a3a6-4bf9-b55f-4c26c1b39832 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:00.446567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:00.446567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:00.446567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:00.446567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:00.446567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:00.446567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:00.446567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:00.446567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:00.446567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:00.446567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:00.446567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:00.446567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:00.446567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:00.446567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:00.447601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:00.447601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:00.447601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:00.447601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:00.447601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:00.447601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:00.447601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:00.447601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:00.447601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:00.447601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:00.447601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:00.447601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:00.447601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:00.447601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:00.447601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:00.447601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:00.447601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:00.448474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:00.448474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:00.448474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:00.448474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:00.448474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 292/560] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:00.449253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d6532-77fe-4aaa-97f1-95d2b2c21c17 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:00.449253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d6532-77fe-4aaa-97f1-95d2b2c21c17 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:00.449774 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d6532-77fe-4aaa-97f1-95d2b2c21c17 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:00.451485 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 293/561] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:00.451991 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d6532-77fe-4aaa-97f1-95d2b2c21c17 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:00.469681 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:00.522863 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c930e-c412-4e7f-8c68-08498d31171c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:00.524752 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:00.560229 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c752b-cec1-4b40-942c-0f19bfe7fedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:00.561333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c752b-cec1-4b40-942c-0f19bfe7fedc None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:00.561614 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c752b-cec1-4b40-942c-0f19bfe7fedc None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:00.561891 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c752b-cec1-4b40-942c-0f19bfe7fedc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:00.569007 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d6532-77fe-4aaa-97f1-95d2b2c21c17 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:00.572954 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c752b-cec1-4b40-942c-0f19bfe7fedc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:00.581104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-1a2c752b-cec1-4b40-942c-0f19bfe7fedc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:00.581104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:00.581104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:00.581104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:00.581104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:00.581104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:00.581104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:00.581104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:00.581104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:00.581104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:00.581104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:00.581104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:00.581104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:00.581104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:00.581104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:00.581104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:00.582033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:00.582033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:00.582033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:00.582033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:00.582033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:00.582033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:00.582033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:00.582033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:00.582033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:00.582033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:00.582033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:00.582033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:00.582033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:00.582033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:00.582033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:00.582033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:00.583339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:00.583339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:00.583339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:00.583339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:00.583339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:00.583339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:00.583339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:00.583339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:00.583339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:00.583339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:00.583339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:00.583339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:00.583339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 269/562] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:26:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:00.583339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c752b-cec1-4b40-942c-0f19bfe7fedc None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:00.610875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f50d1-36cf-4e71-92b9-a825f5186a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:00.611592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f50d1-36cf-4e71-92b9-a825f5186a76 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:00.611886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f50d1-36cf-4e71-92b9-a825f5186a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:00.612167 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f50d1-36cf-4e71-92b9-a825f5186a76 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:00.613009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f50d1-36cf-4e71-92b9-a825f5186a76 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:00.613322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f50d1-36cf-4e71-92b9-a825f5186a76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:00.614800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f50d1-36cf-4e71-92b9-a825f5186a76 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:00.636822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 294/563] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jun 23 11:26:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:00.636822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e57c0-6fad-495c-b32d-061a5d29b0a3 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:00.636822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e57c0-6fad-495c-b32d-061a5d29b0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:00.636822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e57c0-6fad-495c-b32d-061a5d29b0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:00.650917 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-9e4e57c0-6fad-495c-b32d-061a5d29b0a3 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:00.652087 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-9e4e57c0-6fad-495c-b32d-061a5d29b0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:00.696712 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c752b-cec1-4b40-942c-0f19bfe7fedc None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:00.712780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:00.725326 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 295/564] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:26:00.726307 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e57c0-6fad-495c-b32d-061a5d29b0a3 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:00.743361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b04817-4696-4b49-9005-40f19c91cf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:00.744391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b04817-4696-4b49-9005-40f19c91cf13 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:00.744883 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b04817-4696-4b49-9005-40f19c91cf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:00.745329 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b04817-4696-4b49-9005-40f19c91cf13 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:00.746286 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b04817-4696-4b49-9005-40f19c91cf13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:00.746928 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b04817-4696-4b49-9005-40f19c91cf13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:00.748813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b04817-4696-4b49-9005-40f19c91cf13 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:00.762359 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 270/565] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:00.763988 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b04817-4696-4b49-9005-40f19c91cf13 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:00.790813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:00.843011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e57c0-6fad-495c-b32d-061a5d29b0a3 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:00.867698 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b04817-4696-4b49-9005-40f19c91cf13 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:00.876570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e88a57-8118-4000-bc8c-9cacf23527e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:00.877236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e88a57-8118-4000-bc8c-9cacf23527e1 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:00.877514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e88a57-8118-4000-bc8c-9cacf23527e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:00.877782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e88a57-8118-4000-bc8c-9cacf23527e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:00.878517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e88a57-8118-4000-bc8c-9cacf23527e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:00.878852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e88a57-8118-4000-bc8c-9cacf23527e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:00.880212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e88a57-8118-4000-bc8c-9cacf23527e1 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:00.888095 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 296/566] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:26:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:00.889416 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e88a57-8118-4000-bc8c-9cacf23527e1 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:00.902495 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144e84c-82d6-44b4-a773-ef168bb537cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:00.903309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144e84c-82d6-44b4-a773-ef168bb537cd None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:00.903647 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144e84c-82d6-44b4-a773-ef168bb537cd None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:00.904015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144e84c-82d6-44b4-a773-ef168bb537cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:00.913891 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144e84c-82d6-44b4-a773-ef168bb537cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:00.915375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144e84c-82d6-44b4-a773-ef168bb537cd None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:00.932324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-0144e84c-82d6-44b4-a773-ef168bb537cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:00.932324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:00.932324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:00.932324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:00.932324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:00.932324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:00.932324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:00.932324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:00.932324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:00.932324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:00.932324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:00.932324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:00.932324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:00.932324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:00.932324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:00.933213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:00.933213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:00.933213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:00.933213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:00.933213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:00.933213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:00.933213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:00.933213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:00.933213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:00.933213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:00.933213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:00.933213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:00.933213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:00.933213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:00.933213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:00.933213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:00.933213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:00.934095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:00.934095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:00.934095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:00.934095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:00.934095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 271/567] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:00.934376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144e84c-82d6-44b4-a773-ef168bb537cd None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:00.961978 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:00.993033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e88a57-8118-4000-bc8c-9cacf23527e1 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:01.006274 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:01.027151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b542d0-2b59-4b0e-aeed-95570b7ab5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:01.027949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b542d0-2b59-4b0e-aeed-95570b7ab5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:01.028234 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b542d0-2b59-4b0e-aeed-95570b7ab5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:01.028512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b542d0-2b59-4b0e-aeed-95570b7ab5a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:01.034601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144e84c-82d6-44b4-a773-ef168bb537cd None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:01.039120 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b542d0-2b59-4b0e-aeed-95570b7ab5a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:01.047314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-32b542d0-2b59-4b0e-aeed-95570b7ab5a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:01.047314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:01.047314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:01.047314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:01.047314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:01.047314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:01.047314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:01.047314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:01.047314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:01.047314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:01.047314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:01.047314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:01.047314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:01.047314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:01.047314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:01.047314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:01.048418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:01.048418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:01.048418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:01.048418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:01.048418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:01.048418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:01.048418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:01.048418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:01.048418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:01.048418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:01.048418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:01.048418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:01.048418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:01.048418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:01.048418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:01.048418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:01.049442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:01.049442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:01.049442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:01.049442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:01.049442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:01.049442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:01.049442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:01.049442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:01.049442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:01.049442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:01.049442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:01.049442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:01.054828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 297/568] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:26:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:01.055941 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee076b-58c9-47c2-9dc1-0d4e4c4898ee None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:01.056232 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee076b-58c9-47c2-9dc1-0d4e4c4898ee None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:01.056760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee076b-58c9-47c2-9dc1-0d4e4c4898ee None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:01.057802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 298/569] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:01.058485 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee076b-58c9-47c2-9dc1-0d4e4c4898ee None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:01.072540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee44778-6220-4256-a37d-eaa396d21589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:01.073327 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee44778-6220-4256-a37d-eaa396d21589 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:01.073608 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee44778-6220-4256-a37d-eaa396d21589 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:01.073877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee44778-6220-4256-a37d-eaa396d21589 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:01.081064 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee44778-6220-4256-a37d-eaa396d21589 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:01.082497 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee44778-6220-4256-a37d-eaa396d21589 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:01.095888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-cee44778-6220-4256-a37d-eaa396d21589 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:01.095888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:01.095888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:01.095888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:01.095888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:01.095888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:01.095888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:01.095888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:01.095888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:01.095888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:01.095888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:01.095888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:01.095888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:01.095888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:01.095888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:01.096942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:01.096942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:01.096942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:01.096942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:01.096942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:01.096942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:01.096942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:01.096942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:01.096942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:01.096942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:01.096942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:01.096942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:01.096942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:01.096942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:01.096942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:01.096942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:01.096942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:01.097906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:01.097906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:01.097906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:01.097906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:01.097906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:01.097906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:01.097906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:01.097906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:01.097906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 272/570] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:01.097906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee44778-6220-4256-a37d-eaa396d21589 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:01.161829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee076b-58c9-47c2-9dc1-0d4e4c4898ee None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:01.181170 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:01.196357 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695589b0-c369-4438-9f44-3090a71afd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:01.196652 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee44778-6220-4256-a37d-eaa396d21589 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:01.197144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695589b0-c369-4438-9f44-3090a71afd28 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:01.197427 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695589b0-c369-4438-9f44-3090a71afd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:01.197702 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695589b0-c369-4438-9f44-3090a71afd28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:01.207973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695589b0-c369-4438-9f44-3090a71afd28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:01.209504 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695589b0-c369-4438-9f44-3090a71afd28 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:01.226749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-695589b0-c369-4438-9f44-3090a71afd28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:01.226749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:01.226749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:01.226749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:01.226749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:01.226749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:01.226749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:01.226749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:01.226749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:01.226749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:01.226749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:01.226749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:01.226749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:01.226749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:01.226749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:01.227809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:01.227809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:01.227809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:01.227809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:01.227809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:01.227809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:01.227809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:01.227809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:01.227809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:01.227809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:01.227809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:01.227809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:01.227809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:01.227809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:01.227809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:01.227809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:01.227809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:01.228986 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:01.228986 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:01.228986 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:01.228986 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:01.228986 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 299/571] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:01.228986 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695589b0-c369-4438-9f44-3090a71afd28 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:01.232380 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c7204-68da-4dda-af3c-0c5ca317eb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:01.233190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c7204-68da-4dda-af3c-0c5ca317eb09 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:01.233481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c7204-68da-4dda-af3c-0c5ca317eb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:01.233759 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c7204-68da-4dda-af3c-0c5ca317eb09 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:01.234483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c7204-68da-4dda-af3c-0c5ca317eb09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:01.234856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c7204-68da-4dda-af3c-0c5ca317eb09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:01.236215 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c7204-68da-4dda-af3c-0c5ca317eb09 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:01.248442 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 273/572] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:01.249559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6385d40-e018-49cc-940d-6f18b7264bf7 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:01.249845 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6385d40-e018-49cc-940d-6f18b7264bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:01.250121 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6385d40-e018-49cc-940d-6f18b7264bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:01.260006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:01.266702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b6385d40-e018-49cc-940d-6f18b7264bf7 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:01.267890 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b6385d40-e018-49cc-940d-6f18b7264bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:01.334305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695589b0-c369-4438-9f44-3090a71afd28 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:01.335427 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 274/573] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:26:01.336250 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6385d40-e018-49cc-940d-6f18b7264bf7 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:01.371247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a8d29-ca6f-4a95-9d4a-e66b20e85534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:01.371837 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a8d29-ca6f-4a95-9d4a-e66b20e85534 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:01.372122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a8d29-ca6f-4a95-9d4a-e66b20e85534 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:01.372429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a8d29-ca6f-4a95-9d4a-e66b20e85534 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:01.373617 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a8d29-ca6f-4a95-9d4a-e66b20e85534 None admin] RBAC: Adding query filter params (name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:01.373925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a8d29-ca6f-4a95-9d4a-e66b20e85534 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:01.375313 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a8d29-ca6f-4a95-9d4a-e66b20e85534 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:01.383031 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 300/574] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:26:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:01.383809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a8d29-ca6f-4a95-9d4a-e66b20e85534 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:01.424565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:01.447236 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6385d40-e018-49cc-940d-6f18b7264bf7 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:01.469104 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:01.481308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e3a0d-8e2d-4fe9-b2cf-96fba496715e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:01.482729 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e3a0d-8e2d-4fe9-b2cf-96fba496715e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:01.483021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e3a0d-8e2d-4fe9-b2cf-96fba496715e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:01.483300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e3a0d-8e2d-4fe9-b2cf-96fba496715e None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:01.483897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a8d29-ca6f-4a95-9d4a-e66b20e85534 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:01.484110 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e3a0d-8e2d-4fe9-b2cf-96fba496715e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:01.484421 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e3a0d-8e2d-4fe9-b2cf-96fba496715e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:01.485924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e3a0d-8e2d-4fe9-b2cf-96fba496715e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:01.500236 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 275/575] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:01.501058 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e3a0d-8e2d-4fe9-b2cf-96fba496715e None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:01.520360 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95422baf-b1cf-4f4d-80a9-897d4ada560b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:01.521118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95422baf-b1cf-4f4d-80a9-897d4ada560b None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:01.521466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95422baf-b1cf-4f4d-80a9-897d4ada560b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:01.521827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95422baf-b1cf-4f4d-80a9-897d4ada560b None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:01.522645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95422baf-b1cf-4f4d-80a9-897d4ada560b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:01.522952 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95422baf-b1cf-4f4d-80a9-897d4ada560b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:01.524325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95422baf-b1cf-4f4d-80a9-897d4ada560b None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:01.535380 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 301/576] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:01.536145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95422baf-b1cf-4f4d-80a9-897d4ada560b None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:01.600949 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e3a0d-8e2d-4fe9-b2cf-96fba496715e None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:01.638790 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd89eb9-ca33-4a9e-9044-3884c5e5f849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:01.639001 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95422baf-b1cf-4f4d-80a9-897d4ada560b None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:01.639508 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd89eb9-ca33-4a9e-9044-3884c5e5f849 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:01.639792 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd89eb9-ca33-4a9e-9044-3884c5e5f849 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:01.640065 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd89eb9-ca33-4a9e-9044-3884c5e5f849 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:01.642734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:01.647074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd89eb9-ca33-4a9e-9044-3884c5e5f849 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:01.651678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd89eb9-ca33-4a9e-9044-3884c5e5f849 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:01.663569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-2cd89eb9-ca33-4a9e-9044-3884c5e5f849 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:01.663569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:01.663569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:01.663569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:01.663569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:01.663569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:01.663569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:01.663569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:01.663569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:01.663569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:01.663569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:01.663569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:01.663569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:01.663569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:01.663569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:01.664461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:01.664461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:01.664461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:01.664461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:01.664461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:01.664461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:01.664461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:01.664461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:01.664461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:01.664461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:01.664461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:01.664461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:01.664461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:01.664461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:01.664461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:01.664461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:01.664461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:01.665333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:01.665333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:01.665333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:01.665333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:01.665333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:01.665333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:01.665333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:01.665333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:01.665333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 276/577] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:01.666326 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd89eb9-ca33-4a9e-9044-3884c5e5f849 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:01.677141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa13d9-573c-4b37-92d9-d1c8c11e1dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:01.677930 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa13d9-573c-4b37-92d9-d1c8c11e1dbe None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:01.678208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa13d9-573c-4b37-92d9-d1c8c11e1dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:01.678475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa13d9-573c-4b37-92d9-d1c8c11e1dbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:01.689243 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa13d9-573c-4b37-92d9-d1c8c11e1dbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:01.697003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-3baa13d9-573c-4b37-92d9-d1c8c11e1dbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:01.697003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:01.697003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:01.697003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:01.697003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:01.697003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:01.697003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:01.697003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:01.697003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:01.697003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:01.697003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:01.697003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:01.697003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:01.697003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:01.697003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:01.697003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:01.697897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:01.697897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:01.697897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:01.697897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:01.697897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:01.697897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:01.697897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:01.697897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:01.697897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:01.697897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:01.697897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:01.697897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:01.697897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:01.697897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:01.697897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:01.697897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:01.699023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:01.699023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:01.699023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:01.699023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:01.699023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:01.699023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:01.699023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:01.699023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:01.699023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:01.699023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:01.699023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:01.699023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:01.699023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 302/578] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:26:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:01.699993 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa13d9-573c-4b37-92d9-d1c8c11e1dbe None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:01.708994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:01.775970 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd89eb9-ca33-4a9e-9044-3884c5e5f849 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:01.799841 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa13d9-573c-4b37-92d9-d1c8c11e1dbe None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:01.810740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf9601-3819-4e81-8494-237917fbce5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:01.811537 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf9601-3819-4e81-8494-237917fbce5f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:01.811821 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf9601-3819-4e81-8494-237917fbce5f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:01.812100 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf9601-3819-4e81-8494-237917fbce5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:01.822195 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf9601-3819-4e81-8494-237917fbce5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:01.823687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf9601-3819-4e81-8494-237917fbce5f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:01.832856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38baae90-dca6-4a90-b3d6-9008efac814e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:01.833662 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38baae90-dca6-4a90-b3d6-9008efac814e None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:01.833946 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38baae90-dca6-4a90-b3d6-9008efac814e None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:01.834223 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38baae90-dca6-4a90-b3d6-9008efac814e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:01.835087 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38baae90-dca6-4a90-b3d6-9008efac814e None admin] RBAC: Adding query filter params (role.id=13708a4819a94b94b47ba5994aaa9310, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=450b1fad96604fb09d0705895c6d3da4) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:01.835398 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38baae90-dca6-4a90-b3d6-9008efac814e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:01.836805 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38baae90-dca6-4a90-b3d6-9008efac814e None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:01.841582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-1adf9601-3819-4e81-8494-237917fbce5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:01.841582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:01.841582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:01.841582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:01.841582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:01.841582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:01.841582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:01.841582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:01.841582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:01.841582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:01.841582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:01.841582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:01.841582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:01.841582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:01.841582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:01.842711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:01.842711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:01.842711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:01.842711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:01.842711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:01.842711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:01.842711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:01.842711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:01.842711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:01.842711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:01.842711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:01.842711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:01.842711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:01.842711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:01.842711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:01.842711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:01.842711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:01.848237 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:01.848237 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:01.848237 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:01.848237 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:01.848237 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 277/579] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:01.848237 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf9601-3819-4e81-8494-237917fbce5f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:01.850878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 303/580] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:26:01 2022] GET /identity/v3/role_assignments?user.id=450b1fad96604fb09d0705895c6d3da4&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=13708a4819a94b94b47ba5994aaa9310 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:01.851723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38baae90-dca6-4a90-b3d6-9008efac814e None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:01.905661 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:01.944228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:01.975722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf9601-3819-4e81-8494-237917fbce5f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:01.982774 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38baae90-dca6-4a90-b3d6-9008efac814e None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:02.016483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09577c4f-4a85-414f-8e1d-b5f045a38768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:02.017293 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09577c4f-4a85-414f-8e1d-b5f045a38768 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:02.017580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09577c4f-4a85-414f-8e1d-b5f045a38768 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:02.017854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09577c4f-4a85-414f-8e1d-b5f045a38768 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:02.019208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d070b6-e968-4912-a72c-83d1331e8263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:02.019868 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d070b6-e968-4912-a72c-83d1331e8263 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:02.020151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d070b6-e968-4912-a72c-83d1331e8263 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:02.020427 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d070b6-e968-4912-a72c-83d1331e8263 None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:02.021210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d070b6-e968-4912-a72c-83d1331e8263 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:02.021517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d070b6-e968-4912-a72c-83d1331e8263 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:02.022986 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d070b6-e968-4912-a72c-83d1331e8263 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:02.025864 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09577c4f-4a85-414f-8e1d-b5f045a38768 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:02.027382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09577c4f-4a85-414f-8e1d-b5f045a38768 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:02.040561 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 304/581] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:02.040561 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d070b6-e968-4912-a72c-83d1331e8263 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:02.051451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-09577c4f-4a85-414f-8e1d-b5f045a38768 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 11:26:02.051451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:02.051451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:02.051451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:02.051451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:02.051451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:02.051451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:02.051451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:02.051451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:02.051451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:02.051451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:02.051451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:02.051451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:02.051451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:02.051451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:02.051451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:02.052435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:02.052435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:02.052435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:02.052435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:02.052435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:02.052435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:02.052435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:02.052435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:02.052435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:02.052435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:02.052435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:02.052435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:02.052435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:02.052435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:02.052435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:02.052435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:02.052435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:02.052435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:02.053394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:02.053394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:02.053394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:02.053394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:02.053394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:02.053394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:02.053394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:02.053394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:02.053394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 11:26:02.053394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:02.054682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 278/582] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:26:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:02.055486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09577c4f-4a85-414f-8e1d-b5f045a38768 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:02.147662 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d070b6-e968-4912-a72c-83d1331e8263 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:02.163731 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09577c4f-4a85-414f-8e1d-b5f045a38768 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:02.173270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:02.179963 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:02.185799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c2fc0-40ea-45ec-82bb-6ef338527de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:02.186667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c2fc0-40ea-45ec-82bb-6ef338527de6 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:02.187096 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c2fc0-40ea-45ec-82bb-6ef338527de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:02.188338 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c2fc0-40ea-45ec-82bb-6ef338527de6 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:02.188545 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c2fc0-40ea-45ec-82bb-6ef338527de6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:02.189076 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c2fc0-40ea-45ec-82bb-6ef338527de6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:02.190647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c2fc0-40ea-45ec-82bb-6ef338527de6 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:02.198010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99869bfa-4766-42b4-9bb5-398971af1fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:02.198730 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99869bfa-4766-42b4-9bb5-398971af1fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:02.199020 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99869bfa-4766-42b4-9bb5-398971af1fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:02.199294 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99869bfa-4766-42b4-9bb5-398971af1fb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:02.200116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99869bfa-4766-42b4-9bb5-398971af1fb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:02.200265 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 305/583] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:26:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:02.200915 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99869bfa-4766-42b4-9bb5-398971af1fb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:02.201986 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c2fc0-40ea-45ec-82bb-6ef338527de6 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:02.202520 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99869bfa-4766-42b4-9bb5-398971af1fb1 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:02.214602 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 279/584] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:02.215474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99869bfa-4766-42b4-9bb5-398971af1fb1 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:02.314805 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c2fc0-40ea-45ec-82bb-6ef338527de6 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:02.333060 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99869bfa-4766-42b4-9bb5-398971af1fb1 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:02.350721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a7fd4-27e2-48a1-9392-c347a9a24049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:02.351604 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a7fd4-27e2-48a1-9392-c347a9a24049 None admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:02.351882 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a7fd4-27e2-48a1-9392-c347a9a24049 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:02.352152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a7fd4-27e2-48a1-9392-c347a9a24049 None admin] PATH_INFO: `/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/161c5ed1e47e450290a1f428ee100b68/roles/13708a4819a94b94b47ba5994aaa9310` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:02.367460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a7fd4-27e2-48a1-9392-c347a9a24049 None admin] RBAC: Authorizing `identity:create_grant(project_id=886c2939a14b48febc2e2fe639c8f1e4, user_id=161c5ed1e47e450290a1f428ee100b68, role_id=13708a4819a94b94b47ba5994aaa9310)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:02.369453 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a7fd4-27e2-48a1-9392-c347a9a24049 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:02.371125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0b1f3-2257-4170-9124-04332a12a288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:02.371792 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0b1f3-2257-4170-9124-04332a12a288 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:02.372076 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0b1f3-2257-4170-9124-04332a12a288 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:02.372885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0b1f3-2257-4170-9124-04332a12a288 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:02.374249 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f0b1f3-2257-4170-9124-04332a12a288 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:02.374577 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f0b1f3-2257-4170-9124-04332a12a288 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:02.375998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f0b1f3-2257-4170-9124-04332a12a288 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:02.394059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 280/585] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Jun 23 11:26:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:02.394887 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0b1f3-2257-4170-9124-04332a12a288 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:02.422104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:02.439541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 306/586] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:26:02 2022] PUT /identity/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/161c5ed1e47e450290a1f428ee100b68/roles/13708a4819a94b94b47ba5994aaa9310 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:26:02.440462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a7fd4-27e2-48a1-9392-c347a9a24049 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:02.511896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:02.533924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0b1f3-2257-4170-9124-04332a12a288 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:02.577910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f28af0-6f5f-4e7a-915a-8d433ea69917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:02.579164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f28af0-6f5f-4e7a-915a-8d433ea69917 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:02.579762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f28af0-6f5f-4e7a-915a-8d433ea69917 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:02.580289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f28af0-6f5f-4e7a-915a-8d433ea69917 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:02.591694 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a7fd4-27e2-48a1-9392-c347a9a24049 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:02.595515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f28af0-6f5f-4e7a-915a-8d433ea69917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:02.597557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f28af0-6f5f-4e7a-915a-8d433ea69917 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:02.619059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-06f28af0-6f5f-4e7a-915a-8d433ea69917 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:02.619059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:02.619059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:02.619059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:02.619059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:02.619059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:02.619059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:02.619059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:02.619059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:02.619059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:02.619059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:02.619059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:02.619059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:02.619059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:02.619059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:02.620015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:02.620015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:02.620015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:02.620015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:02.620015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:02.620015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:02.620015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:02.620015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:02.620015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:02.620015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:02.620015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:02.620015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:02.620015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:02.620015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:02.620015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:02.620015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:02.620015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:02.620907 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:02.620907 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:02.620907 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:02.620907 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:02.621560 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 281/587] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:02.622115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f28af0-6f5f-4e7a-915a-8d433ea69917 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:02.636318 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917173eb-2cd7-4f34-81b6-35f174f3f0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:02.637116 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917173eb-2cd7-4f34-81b6-35f174f3f0ef None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:02.637393 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917173eb-2cd7-4f34-81b6-35f174f3f0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:02.637699 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917173eb-2cd7-4f34-81b6-35f174f3f0ef None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:02.648644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917173eb-2cd7-4f34-81b6-35f174f3f0ef None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:02.650438 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917173eb-2cd7-4f34-81b6-35f174f3f0ef None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:02.671093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-917173eb-2cd7-4f34-81b6-35f174f3f0ef None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 11:26:02.671093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:02.671093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:02.671093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:02.671093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:02.671093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:02.671093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:02.671093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:02.671093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:02.671093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:02.671093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:02.671093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:02.671093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:02.671093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:02.671093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:02.671093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:02.672322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:02.672322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:02.672322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:02.672322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:02.672322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:02.672322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:02.672322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:02.672322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:02.672322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:02.672322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:02.672322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:02.672322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:02.672322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:02.672322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:02.672322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:02.672322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:02.672322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:02.672322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:02.673670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:02.673670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:02.673670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:02.673670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:02.673670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:02.673670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:02.673670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:02.673670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:02.673670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 11:26:02.673670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:02.676036 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 307/588] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:26:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:02.680997 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917173eb-2cd7-4f34-81b6-35f174f3f0ef None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:02.734451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:02.764104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f28af0-6f5f-4e7a-915a-8d433ea69917 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:02.786549 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917173eb-2cd7-4f34-81b6-35f174f3f0ef None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:02.790869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:02.811516 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1981c4f-25fa-4331-bad5-8e69cdf05b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:02.813060 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1981c4f-25fa-4331-bad5-8e69cdf05b89 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:02.813812 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1981c4f-25fa-4331-bad5-8e69cdf05b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:02.814275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1981c4f-25fa-4331-bad5-8e69cdf05b89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:02.828761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce96e48-4043-47eb-885e-5c9b5408325c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:02.829434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce96e48-4043-47eb-885e-5c9b5408325c None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:02.829612 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1981c4f-25fa-4331-bad5-8e69cdf05b89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:02.829742 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce96e48-4043-47eb-885e-5c9b5408325c None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:02.829982 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce96e48-4043-47eb-885e-5c9b5408325c None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:02.830749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce96e48-4043-47eb-885e-5c9b5408325c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:02.831062 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce96e48-4043-47eb-885e-5c9b5408325c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:02.832056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1981c4f-25fa-4331-bad5-8e69cdf05b89 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:02.832515 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce96e48-4043-47eb-885e-5c9b5408325c None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:02.847424 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 308/589] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:02.848349 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce96e48-4043-47eb-885e-5c9b5408325c None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:02.857834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-e1981c4f-25fa-4331-bad5-8e69cdf05b89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:02.857834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:02.857834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:02.857834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:02.857834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:02.857834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:02.857834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:02.857834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:02.857834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:02.857834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:02.857834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:02.857834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:02.857834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:02.857834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:02.857834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:02.858894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:02.858894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:02.858894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:02.858894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:02.858894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:02.858894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:02.858894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:02.858894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:02.858894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:02.858894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:02.858894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:02.858894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:02.858894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:02.858894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:02.858894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:02.858894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:02.858894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:02.859830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:02.859830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:02.859830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:02.859830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:02.860827 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 282/590] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:02.861419 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1981c4f-25fa-4331-bad5-8e69cdf05b89 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:02.968279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce96e48-4043-47eb-885e-5c9b5408325c None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:02.988150 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1981c4f-25fa-4331-bad5-8e69cdf05b89 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:03.005951 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285056c8-f67b-4697-9e6f-58a0651d3a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:03.006709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285056c8-f67b-4697-9e6f-58a0651d3a7a None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:03.006996 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285056c8-f67b-4697-9e6f-58a0651d3a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:03.007272 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285056c8-f67b-4697-9e6f-58a0651d3a7a None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:03.008119 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285056c8-f67b-4697-9e6f-58a0651d3a7a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:03.008432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285056c8-f67b-4697-9e6f-58a0651d3a7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:03.009943 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285056c8-f67b-4697-9e6f-58a0651d3a7a None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:03.022938 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:03.027136 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b928274-d81f-4e7d-81b1-52a7d6f579a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:03.028063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b928274-d81f-4e7d-81b1-52a7d6f579a5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:03.028349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b928274-d81f-4e7d-81b1-52a7d6f579a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:03.029251 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b928274-d81f-4e7d-81b1-52a7d6f579a5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:03.029782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 309/591] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Jun 23 11:26:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:03.030647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285056c8-f67b-4697-9e6f-58a0651d3a7a None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:03.037723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b928274-d81f-4e7d-81b1-52a7d6f579a5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:03.039288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b928274-d81f-4e7d-81b1-52a7d6f579a5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:03.053339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:03.057726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-3b928274-d81f-4e7d-81b1-52a7d6f579a5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 11:26:03.057726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:03.057726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:03.057726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:03.057726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:03.057726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:03.057726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:03.057726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:03.057726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:03.057726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:03.057726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:03.057726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:03.057726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:03.057726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:03.057726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:03.057726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:03.058678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:03.058678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:03.058678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:03.058678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:03.058678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:03.058678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:03.058678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:03.058678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:03.058678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:03.058678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:03.058678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:03.058678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:03.058678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:03.058678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:03.058678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:03.058678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:03.058678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:03.058678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:03.059623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:03.059623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:03.059623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:03.059623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:03.059623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:03.059623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:03.059623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:03.059623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:03.059623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 11:26:03.059623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:03.064248 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 283/592] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:26:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:03.065329 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b928274-d81f-4e7d-81b1-52a7d6f579a5 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:03.141608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285056c8-f67b-4697-9e6f-58a0651d3a7a None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:03.179633 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aef7a1-4d87-4298-976b-7cad1f46dedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:03.180420 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aef7a1-4d87-4298-976b-7cad1f46dedc None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:03.180776 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aef7a1-4d87-4298-976b-7cad1f46dedc None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:03.181120 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aef7a1-4d87-4298-976b-7cad1f46dedc None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:03.181972 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aef7a1-4d87-4298-976b-7cad1f46dedc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:03.182351 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aef7a1-4d87-4298-976b-7cad1f46dedc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:03.183834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aef7a1-4d87-4298-976b-7cad1f46dedc None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:03.184533 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b928274-d81f-4e7d-81b1-52a7d6f579a5 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:03.197484 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 310/593] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:03.198485 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aef7a1-4d87-4298-976b-7cad1f46dedc None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:03.227250 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea474dff-9a78-4f04-b99c-d68dde003ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:03.228144 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea474dff-9a78-4f04-b99c-d68dde003ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:03.228431 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea474dff-9a78-4f04-b99c-d68dde003ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:03.229024 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea474dff-9a78-4f04-b99c-d68dde003ec9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:03.241943 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea474dff-9a78-4f04-b99c-d68dde003ec9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:03.243806 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea474dff-9a78-4f04-b99c-d68dde003ec9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:03.267519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-ea474dff-9a78-4f04-b99c-d68dde003ec9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:03.267519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:03.267519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:03.267519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:03.267519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:03.267519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:03.267519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:03.267519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:03.267519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:03.267519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:03.267519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:03.267519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:03.267519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:03.267519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:03.267519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:03.269305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:03.269305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:03.269305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:03.269305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:03.269305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:03.269305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:03.269305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:03.269305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:03.269305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:03.269305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:03.269305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:03.269305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:03.269305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:03.269305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:03.269305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:03.269305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:03.269305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:03.271289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:03.271289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:03.271289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:03.271289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:03.271289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 284/594] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:03.271289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea474dff-9a78-4f04-b99c-d68dde003ec9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:03.290085 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:03.309473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aef7a1-4d87-4298-976b-7cad1f46dedc None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:03.330334 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:03.348070 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189f743-bd61-4d2b-b5df-91a643e8a80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:03.348920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189f743-bd61-4d2b-b5df-91a643e8a80d None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:03.349342 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189f743-bd61-4d2b-b5df-91a643e8a80d None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:03.349625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189f743-bd61-4d2b-b5df-91a643e8a80d None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:03.350649 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f189f743-bd61-4d2b-b5df-91a643e8a80d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:03.350970 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f189f743-bd61-4d2b-b5df-91a643e8a80d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:03.352992 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f189f743-bd61-4d2b-b5df-91a643e8a80d None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:03.380564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 311/595] 10.222.0.13 () {56 vars in 1139 bytes} [Thu Jun 23 11:26:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:03.381423 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189f743-bd61-4d2b-b5df-91a643e8a80d None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:03.400118 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea474dff-9a78-4f04-b99c-d68dde003ec9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:03.436629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa526d-036a-4b91-87d1-69358196123b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:03.437428 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa526d-036a-4b91-87d1-69358196123b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:03.437714 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa526d-036a-4b91-87d1-69358196123b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:03.437995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa526d-036a-4b91-87d1-69358196123b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:03.445524 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa526d-036a-4b91-87d1-69358196123b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:03.447094 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa526d-036a-4b91-87d1-69358196123b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:03.461493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-b2aa526d-036a-4b91-87d1-69358196123b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:03.461493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:03.461493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:03.461493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:03.461493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:03.461493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:03.461493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:03.461493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:03.461493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:03.461493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:03.461493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:03.461493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:03.461493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:03.461493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:03.461493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:03.463601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:03.463601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:03.463601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:03.463601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:03.463601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:03.463601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:03.463601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:03.463601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:03.463601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:03.463601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:03.463601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:03.463601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:03.463601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:03.463601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:03.463601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:03.463601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:03.463601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:03.465738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:03.465738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:03.465738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:03.465738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:03.465738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:03.465738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:03.465738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:03.465738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:03.466917 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 285/596] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:03.470863 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa526d-036a-4b91-87d1-69358196123b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:03.507142 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189f743-bd61-4d2b-b5df-91a643e8a80d None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:03.553713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61cc801-5cc8-43d3-b260-1519d300a564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:03.554409 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cc801-5cc8-43d3-b260-1519d300a564 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:03.555293 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cc801-5cc8-43d3-b260-1519d300a564 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:03.555580 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cc801-5cc8-43d3-b260-1519d300a564 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:03.556321 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61cc801-5cc8-43d3-b260-1519d300a564 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:03.556630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61cc801-5cc8-43d3-b260-1519d300a564 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:03.558562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61cc801-5cc8-43d3-b260-1519d300a564 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:03.567222 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:03.574289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 312/597] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:03.575654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61cc801-5cc8-43d3-b260-1519d300a564 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:03.601152 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa526d-036a-4b91-87d1-69358196123b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:03.603190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:03.650042 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43aef7-aaf5-43f5-9618-ea57e20f42fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:03.651814 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43aef7-aaf5-43f5-9618-ea57e20f42fa None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:03.652278 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43aef7-aaf5-43f5-9618-ea57e20f42fa None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:03.653001 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43aef7-aaf5-43f5-9618-ea57e20f42fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:03.668337 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43aef7-aaf5-43f5-9618-ea57e20f42fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:03.670800 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43aef7-aaf5-43f5-9618-ea57e20f42fa None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:03.695491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-9b43aef7-aaf5-43f5-9618-ea57e20f42fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:03.695491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:03.695491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:03.695491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:03.695491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:03.695491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:03.695491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:03.695491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:03.695491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:03.695491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:03.695491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:03.695491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:03.695491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:03.695491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:03.695491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:03.696592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:03.696592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:03.696592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:03.696592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:03.696592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:03.696592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:03.696592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:03.696592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:03.696592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:03.696592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:03.696592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:03.696592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:03.696592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:03.696592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:03.696592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:03.696592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:03.696592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:03.697770 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:03.697770 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:03.697770 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:03.697770 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:03.700554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 286/598] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:03.701617 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43aef7-aaf5-43f5-9618-ea57e20f42fa None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:03.709858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61cc801-5cc8-43d3-b260-1519d300a564 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:03.753254 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc6ab7-388c-4f8d-a485-5cf761b5a762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:03.754459 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc6ab7-388c-4f8d-a485-5cf761b5a762 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:03.754879 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc6ab7-388c-4f8d-a485-5cf761b5a762 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:03.755254 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc6ab7-388c-4f8d-a485-5cf761b5a762 None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:03.756202 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc6ab7-388c-4f8d-a485-5cf761b5a762 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:03.756613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc6ab7-388c-4f8d-a485-5cf761b5a762 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:03.758987 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc6ab7-388c-4f8d-a485-5cf761b5a762 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:03.775036 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 313/599] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:03.776129 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc6ab7-388c-4f8d-a485-5cf761b5a762 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:03.838174 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43aef7-aaf5-43f5-9618-ea57e20f42fa None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:03.850581 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:03.883472 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e2593-8d3b-4022-9156-18cf823b8d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:03.884582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e2593-8d3b-4022-9156-18cf823b8d4f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:03.884872 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e2593-8d3b-4022-9156-18cf823b8d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:03.885151 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e2593-8d3b-4022-9156-18cf823b8d4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:03.894611 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e2593-8d3b-4022-9156-18cf823b8d4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:03.896626 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e2593-8d3b-4022-9156-18cf823b8d4f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:03.906202 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc6ab7-388c-4f8d-a485-5cf761b5a762 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:03.912439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-df9e2593-8d3b-4022-9156-18cf823b8d4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:03.912439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:03.912439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:03.912439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:03.912439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:03.912439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:03.912439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:03.912439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:03.912439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:03.912439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:03.912439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:03.912439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:03.912439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:03.912439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:03.912439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:03.913338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:03.913338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:03.913338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:03.913338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:03.913338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:03.913338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:03.913338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:03.913338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:03.913338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:03.913338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:03.913338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:03.913338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:03.913338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:03.913338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:03.913338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:03.913338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:03.913338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:03.914231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:03.914231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:03.914231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:03.914231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:03.914231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:03.914231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:03.914231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:03.914231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:03.916840 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 287/600] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:03.917923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e2593-8d3b-4022-9156-18cf823b8d4f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:03.936081 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:03.943978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d8a5f-1205-40a4-81fd-e58648d7f519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:03.944889 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d8a5f-1205-40a4-81fd-e58648d7f519 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:03.949360 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d8a5f-1205-40a4-81fd-e58648d7f519 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:03.949360 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d8a5f-1205-40a4-81fd-e58648d7f519 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:03.949360 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d8a5f-1205-40a4-81fd-e58648d7f519 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:03.949360 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d8a5f-1205-40a4-81fd-e58648d7f519 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:03.949360 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d8a5f-1205-40a4-81fd-e58648d7f519 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:03.961300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 314/601] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:03.962464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d8a5f-1205-40a4-81fd-e58648d7f519 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:04.033554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e2593-8d3b-4022-9156-18cf823b8d4f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:04.067243 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db02790-55eb-488c-b205-feba84a9058b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:04.068394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db02790-55eb-488c-b205-feba84a9058b None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:04.068676 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db02790-55eb-488c-b205-feba84a9058b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:04.068799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d8a5f-1205-40a4-81fd-e58648d7f519 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:04.069213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db02790-55eb-488c-b205-feba84a9058b None admin] PATH_INFO: `/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/b6105838cbf1409587ee7d9e3be554f8/roles/13708a4819a94b94b47ba5994aaa9310` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:04.084181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db02790-55eb-488c-b205-feba84a9058b None admin] RBAC: Authorizing `identity:create_grant(project_id=886c2939a14b48febc2e2fe639c8f1e4, user_id=b6105838cbf1409587ee7d9e3be554f8, role_id=13708a4819a94b94b47ba5994aaa9310)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:04.085618 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db02790-55eb-488c-b205-feba84a9058b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:04.086887 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:04.104238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24ad64-6100-4b2d-8384-4bc15dc7005e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:04.104894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24ad64-6100-4b2d-8384-4bc15dc7005e None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:04.105187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24ad64-6100-4b2d-8384-4bc15dc7005e None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:04.105466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24ad64-6100-4b2d-8384-4bc15dc7005e None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:04.106251 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24ad64-6100-4b2d-8384-4bc15dc7005e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:04.107201 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24ad64-6100-4b2d-8384-4bc15dc7005e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:04.108714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24ad64-6100-4b2d-8384-4bc15dc7005e None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:04.120798 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 315/602] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:04.121582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24ad64-6100-4b2d-8384-4bc15dc7005e None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:04.159631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 288/603] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:26:03 2022] PUT /identity/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/b6105838cbf1409587ee7d9e3be554f8/roles/13708a4819a94b94b47ba5994aaa9310 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:26:04.161261 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd2713-6e05-4684-9e00-448ccdb5e4c7 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:04.161709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd2713-6e05-4684-9e00-448ccdb5e4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:04.162155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd2713-6e05-4684-9e00-448ccdb5e4c7 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:04.164201 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 289/604] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:04.165271 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd2713-6e05-4684-9e00-448ccdb5e4c7 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:04.247163 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:04.261582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24ad64-6100-4b2d-8384-4bc15dc7005e None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:04.296557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd2713-6e05-4684-9e00-448ccdb5e4c7 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:04.315199 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedf3af-4127-4c2a-8cef-af6d27be7529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:04.317415 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedf3af-4127-4c2a-8cef-af6d27be7529 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:04.317951 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedf3af-4127-4c2a-8cef-af6d27be7529 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:04.318462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedf3af-4127-4c2a-8cef-af6d27be7529 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:04.331177 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedf3af-4127-4c2a-8cef-af6d27be7529 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:04.333911 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedf3af-4127-4c2a-8cef-af6d27be7529 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:04.345112 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea597b6d-ec82-473e-9ab0-79648a90bb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:04.346015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea597b6d-ec82-473e-9ab0-79648a90bb61 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:04.346302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea597b6d-ec82-473e-9ab0-79648a90bb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:04.346683 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea597b6d-ec82-473e-9ab0-79648a90bb61 None admin] PATH_INFO: `/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/871dd684032c49d09608f0af05b7a765/roles/13708a4819a94b94b47ba5994aaa9310` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:04.352593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-1eedf3af-4127-4c2a-8cef-af6d27be7529 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:04.352593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:04.352593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:04.352593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:04.352593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:04.352593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:04.352593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:04.352593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:04.352593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:04.352593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:04.352593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:04.352593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:04.352593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:04.352593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:04.352593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:04.353474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:04.353474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:04.353474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:04.353474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:04.353474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:04.353474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:04.353474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:04.353474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:04.353474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:04.353474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:04.353474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:04.353474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:04.353474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:04.353474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:04.353474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:04.353474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:04.353474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:04.354371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:04.354371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:04.354371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:04.354371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:04.354371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:04.354371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:04.354371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:04.354371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:04.354371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 316/605] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:04.355324 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c3a19-70f7-45b2-ada8-3997fdea6393 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:04.355405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c3a19-70f7-45b2-ada8-3997fdea6393 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:04.355646 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c3a19-70f7-45b2-ada8-3997fdea6393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:04.363774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea597b6d-ec82-473e-9ab0-79648a90bb61 None admin] RBAC: Authorizing `identity:create_grant(project_id=886c2939a14b48febc2e2fe639c8f1e4, user_id=871dd684032c49d09608f0af05b7a765, role_id=13708a4819a94b94b47ba5994aaa9310)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:04.365236 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea597b6d-ec82-473e-9ab0-79648a90bb61 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:04.373762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-8b0c3a19-70f7-45b2-ada8-3997fdea6393 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:04.376487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-8b0c3a19-70f7-45b2-ada8-3997fdea6393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:04.408903 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:04.446141 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 290/606] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:26:04 2022] PUT /identity/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/871dd684032c49d09608f0af05b7a765/roles/13708a4819a94b94b47ba5994aaa9310 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:26:04.446848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea597b6d-ec82-473e-9ab0-79648a90bb61 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:04.458851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 317/607] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:26:04.481684 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c3a19-70f7-45b2-ada8-3997fdea6393 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:04.650902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea597b6d-ec82-473e-9ab0-79648a90bb61 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:04.651566 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c3a19-70f7-45b2-ada8-3997fdea6393 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:04.668021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:04.697199 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c4e65-24e7-4d17-9c5a-0d6af436c4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:04.698273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c4e65-24e7-4d17-9c5a-0d6af436c4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:04.698391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa0cf2-a672-4bcd-a91c-af71c99f8f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:04.698768 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c4e65-24e7-4d17-9c5a-0d6af436c4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:04.699139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c4e65-24e7-4d17-9c5a-0d6af436c4e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:04.699519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa0cf2-a672-4bcd-a91c-af71c99f8f0e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:04.699959 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa0cf2-a672-4bcd-a91c-af71c99f8f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:04.700480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa0cf2-a672-4bcd-a91c-af71c99f8f0e None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:04.701981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa0cf2-a672-4bcd-a91c-af71c99f8f0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:04.702518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa0cf2-a672-4bcd-a91c-af71c99f8f0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:04.704259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa0cf2-a672-4bcd-a91c-af71c99f8f0e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:04.711281 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c4e65-24e7-4d17-9c5a-0d6af436c4e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:04.718483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 291/608] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:04.720018 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0bc3c-3e28-487c-b560-fa4473edc1b7 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:04.720438 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0bc3c-3e28-487c-b560-fa4473edc1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:04.720867 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0bc3c-3e28-487c-b560-fa4473edc1b7 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:04.722513 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 292/609] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:04.724412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-7b1c4e65-24e7-4d17-9c5a-0d6af436c4e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:04.724412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:04.724412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:04.724412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:04.724412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:04.724412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:04.724412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:04.724412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:04.724412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:04.724412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:04.724412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:04.724412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:04.724412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:04.724412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:04.724412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:04.724412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:04.725511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:04.725511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:04.725511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:04.725511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:04.725511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:04.725511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:04.725511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:04.725511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:04.725511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:04.725511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:04.725511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:04.725511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:04.725511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:04.725511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:04.725511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:04.725511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:04.726964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:04.726964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:04.726964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:04.726964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:04.726964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:04.726964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:04.726964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:04.726964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:04.726964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:04.726964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:04.726964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:04.726964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:04.728613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 318/610] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:26:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:04.729298 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ce598-c647-43e2-b0fa-77cd19c021e4 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:04.729802 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ce598-c647-43e2-b0fa-77cd19c021e4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:04.730138 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c4e65-24e7-4d17-9c5a-0d6af436c4e2 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:04.730390 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ce598-c647-43e2-b0fa-77cd19c021e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:04.748483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-881ce598-c647-43e2-b0fa-77cd19c021e4 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:04.749956 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-881ce598-c647-43e2-b0fa-77cd19c021e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:04.762082 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:04.840864 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 293/611] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:26:04.842846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ce598-c647-43e2-b0fa-77cd19c021e4 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:04.845233 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c4e65-24e7-4d17-9c5a-0d6af436c4e2 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:04.886824 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a431a08-74ee-4444-86c9-587bc8b28412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:04.887709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a431a08-74ee-4444-86c9-587bc8b28412 None admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:04.888790 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a431a08-74ee-4444-86c9-587bc8b28412 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:04.889155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a431a08-74ee-4444-86c9-587bc8b28412 None admin] PATH_INFO: `/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/810fa3f54cd04f989252765f7e76e602/roles/13708a4819a94b94b47ba5994aaa9310` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:04.905398 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a431a08-74ee-4444-86c9-587bc8b28412 None admin] RBAC: Authorizing `identity:create_grant(project_id=886c2939a14b48febc2e2fe639c8f1e4, user_id=810fa3f54cd04f989252765f7e76e602, role_id=13708a4819a94b94b47ba5994aaa9310)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:04.906886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a431a08-74ee-4444-86c9-587bc8b28412 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:04.965186 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ce598-c647-43e2-b0fa-77cd19c021e4 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:04.969313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:04.990564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 319/612] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:26:04 2022] PUT /identity/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/810fa3f54cd04f989252765f7e76e602/roles/13708a4819a94b94b47ba5994aaa9310 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:26:04.990564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a431a08-74ee-4444-86c9-587bc8b28412 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:05.028599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57002594-b4bc-447d-9634-cf1bdfae926c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:05.030731 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57002594-b4bc-447d-9634-cf1bdfae926c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:05.031020 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57002594-b4bc-447d-9634-cf1bdfae926c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:05.032099 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57002594-b4bc-447d-9634-cf1bdfae926c None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:05.032878 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57002594-b4bc-447d-9634-cf1bdfae926c None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:05.033185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57002594-b4bc-447d-9634-cf1bdfae926c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:05.037416 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57002594-b4bc-447d-9634-cf1bdfae926c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:05.059492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 294/613] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:26:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:05.063670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57002594-b4bc-447d-9634-cf1bdfae926c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:05.151297 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:05.181249 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a431a08-74ee-4444-86c9-587bc8b28412 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:05.232537 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57002594-b4bc-447d-9634-cf1bdfae926c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:05.233246 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b5751-0803-4729-ae79-085364b50fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:05.234604 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b5751-0803-4729-ae79-085364b50fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:05.236008 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b5751-0803-4729-ae79-085364b50fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:05.236492 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b5751-0803-4729-ae79-085364b50fe2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:05.249984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b5751-0803-4729-ae79-085364b50fe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:05.262189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-186b5751-0803-4729-ae79-085364b50fe2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:05.262189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:05.262189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:05.262189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:05.262189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:05.262189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:05.262189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:05.262189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:05.262189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:05.262189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:05.262189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:05.262189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:05.262189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:05.262189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:05.262189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:05.262189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:05.263858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:05.263858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:05.263858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:05.263858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:05.263858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:05.263858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:05.263858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:05.263858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:05.263858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:05.263858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:05.263858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:05.263858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:05.263858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:05.263858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:05.263858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:05.263858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:05.264803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:05.264803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:05.264803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:05.264803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:05.264803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:05.264803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:05.264803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:05.264803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:05.264803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:05.264803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:05.264803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:05.264803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:05.267638 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 320/614] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:26:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:05.272894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b5751-0803-4729-ae79-085364b50fe2 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:05.284798 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1644c-4992-459d-9611-c466dce032c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:05.285610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1644c-4992-459d-9611-c466dce032c3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:05.285888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1644c-4992-459d-9611-c466dce032c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:05.286159 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1644c-4992-459d-9611-c466dce032c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:05.305634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1644c-4992-459d-9611-c466dce032c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:05.308786 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1644c-4992-459d-9611-c466dce032c3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:05.346005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-9af1644c-4992-459d-9611-c466dce032c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:05.346005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:05.346005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:05.346005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:05.346005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:05.346005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:05.346005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:05.346005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:05.346005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:05.346005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:05.346005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:05.346005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:05.346005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:05.346005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:05.346005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:05.347000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:05.347000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:05.347000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:05.347000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:05.347000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:05.347000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:05.347000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:05.347000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:05.347000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:05.347000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:05.347000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:05.347000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:05.347000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:05.347000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:05.347000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:05.347000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:05.347000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:05.347879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:05.347879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:05.347879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:05.347879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:05.350130 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 295/615] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:05.355489 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1644c-4992-459d-9611-c466dce032c3 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:05.410841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:05.456694 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b5751-0803-4729-ae79-085364b50fe2 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:05.525536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55edd61-7992-4fec-950b-c20ce0b5f2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:05.526313 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55edd61-7992-4fec-950b-c20ce0b5f2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:05.526626 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55edd61-7992-4fec-950b-c20ce0b5f2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:05.526915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55edd61-7992-4fec-950b-c20ce0b5f2b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:05.527678 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55edd61-7992-4fec-950b-c20ce0b5f2b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:05.528083 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55edd61-7992-4fec-950b-c20ce0b5f2b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:05.530137 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55edd61-7992-4fec-950b-c20ce0b5f2b9 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:05.532057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1644c-4992-459d-9611-c466dce032c3 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:05.546037 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:05.549881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 321/616] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:26:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:05.555030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55edd61-7992-4fec-950b-c20ce0b5f2b9 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:05.569859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ec1f9-2a8f-4f37-b7e7-7493fd05eeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:05.570526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ec1f9-2a8f-4f37-b7e7-7493fd05eeb7 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:05.570829 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ec1f9-2a8f-4f37-b7e7-7493fd05eeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:05.571100 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ec1f9-2a8f-4f37-b7e7-7493fd05eeb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:05.571814 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ec1f9-2a8f-4f37-b7e7-7493fd05eeb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:05.572117 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ec1f9-2a8f-4f37-b7e7-7493fd05eeb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:05.573482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ec1f9-2a8f-4f37-b7e7-7493fd05eeb7 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:05.586563 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 296/617] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:05.592576 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ec1f9-2a8f-4f37-b7e7-7493fd05eeb7 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:05.683000 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55edd61-7992-4fec-950b-c20ce0b5f2b9 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:05.720671 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58050234-876d-4ae4-9205-dab86fda4901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:05.721519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58050234-876d-4ae4-9205-dab86fda4901 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:05.721808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58050234-876d-4ae4-9205-dab86fda4901 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:05.722088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58050234-876d-4ae4-9205-dab86fda4901 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:05.738667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58050234-876d-4ae4-9205-dab86fda4901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:05.738667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58050234-876d-4ae4-9205-dab86fda4901 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:05.765191 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:05.770210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-58050234-876d-4ae4-9205-dab86fda4901 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:05.770210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:05.770210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:05.770210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:05.770210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:05.770210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:05.770210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:05.770210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:05.770210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:05.770210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:05.770210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:05.770210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:05.770210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:05.770210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:05.770210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:05.774670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:05.774670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:05.774670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:05.774670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:05.774670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:05.774670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:05.774670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:05.774670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:05.774670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:05.774670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:05.774670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:05.774670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:05.774670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:05.774670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:05.774670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:05.774670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:05.774670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:05.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:05.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:05.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:05.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:05.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 322/618] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:05.781552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58050234-876d-4ae4-9205-dab86fda4901 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:05.807749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ec1f9-2a8f-4f37-b7e7-7493fd05eeb7 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:05.874459 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7cb48-ecf7-45b9-be32-6113bb6ba05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:05.875304 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7cb48-ecf7-45b9-be32-6113bb6ba05b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:05.875587 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7cb48-ecf7-45b9-be32-6113bb6ba05b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:05.875861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7cb48-ecf7-45b9-be32-6113bb6ba05b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:05.879295 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:05.886094 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7cb48-ecf7-45b9-be32-6113bb6ba05b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:05.887642 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7cb48-ecf7-45b9-be32-6113bb6ba05b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:05.915486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-96d7cb48-ecf7-45b9-be32-6113bb6ba05b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:26:05.915486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:05.915486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:05.915486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:05.915486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:05.915486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:05.915486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:05.915486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:05.915486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:05.915486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:05.915486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:05.915486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:05.915486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:05.915486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:05.915486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:05.915486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:05.916557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:05.916557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:05.916557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:05.916557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:05.916557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:05.916557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:05.916557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:05.916557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:05.916557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:05.916557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:05.916557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:05.916557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:05.916557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:05.916557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:05.916557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:05.916557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:05.916557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:05.916557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:05.917480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:05.917480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:05.917480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:05.917480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:05.917480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:05.917480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:05.917480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:05.917480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:05.917480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:26:05.917480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:05.923851 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 297/619] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:26:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:05.924695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7cb48-ecf7-45b9-be32-6113bb6ba05b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:05.969677 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58050234-876d-4ae4-9205-dab86fda4901 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:06.040392 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130a290-ee82-4c77-a489-e65a22e06d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:06.040984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130a290-ee82-4c77-a489-e65a22e06d26 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:06.041268 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130a290-ee82-4c77-a489-e65a22e06d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:06.041540 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130a290-ee82-4c77-a489-e65a22e06d26 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:06.042270 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130a290-ee82-4c77-a489-e65a22e06d26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:06.042592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130a290-ee82-4c77-a489-e65a22e06d26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:06.044482 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130a290-ee82-4c77-a489-e65a22e06d26 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:06.075407 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 323/620] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:06.084232 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130a290-ee82-4c77-a489-e65a22e06d26 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:06.093241 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7cb48-ecf7-45b9-be32-6113bb6ba05b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:06.154891 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ce31d-ea1f-436a-be5e-0850e43f33a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:06.155592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ce31d-ea1f-436a-be5e-0850e43f33a6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:06.155864 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ce31d-ea1f-436a-be5e-0850e43f33a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:06.156138 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ce31d-ea1f-436a-be5e-0850e43f33a6 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:06.161693 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ce31d-ea1f-436a-be5e-0850e43f33a6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:06.162024 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ce31d-ea1f-436a-be5e-0850e43f33a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:06.163496 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ce31d-ea1f-436a-be5e-0850e43f33a6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:06.179286 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:06.195551 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 298/621] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Jun 23 11:26:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:06.201905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ce31d-ea1f-436a-be5e-0850e43f33a6 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:06.273742 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130a290-ee82-4c77-a489-e65a22e06d26 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:06.288788 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:06.314981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e01b87-44e5-4b88-b640-ff2e805ca801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:06.315955 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e01b87-44e5-4b88-b640-ff2e805ca801 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:06.316393 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e01b87-44e5-4b88-b640-ff2e805ca801 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:06.316670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e01b87-44e5-4b88-b640-ff2e805ca801 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:06.329396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e01b87-44e5-4b88-b640-ff2e805ca801 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:06.338551 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e01b87-44e5-4b88-b640-ff2e805ca801 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:06.359534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-b3e01b87-44e5-4b88-b640-ff2e805ca801 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:26:06.359534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:06.359534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:06.359534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:06.359534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:06.359534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:06.359534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:06.359534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:06.359534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:06.359534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:06.359534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:06.359534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:06.359534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:06.359534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:06.359534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:06.359534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:06.360640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:06.360640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:06.360640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:06.360640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:06.360640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:06.360640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:06.360640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:06.360640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:06.360640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:06.360640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:06.360640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:06.360640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:06.360640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:06.360640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:06.360640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:06.360640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:06.360640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:06.360640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:06.361621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:06.361621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:06.361621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:06.361621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:06.361621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:06.361621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:06.361621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:06.361621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:06.361621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:26:06.361621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:06.373843 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 324/622] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jun 23 11:26:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:06.374835 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e01b87-44e5-4b88-b640-ff2e805ca801 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:06.410878 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ce31d-ea1f-436a-be5e-0850e43f33a6 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:06.488014 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87086608-7fe6-4dc3-90c9-32d96546ce6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:06.489082 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87086608-7fe6-4dc3-90c9-32d96546ce6a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:06.489684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87086608-7fe6-4dc3-90c9-32d96546ce6a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:06.490162 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87086608-7fe6-4dc3-90c9-32d96546ce6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:06.508054 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e01b87-44e5-4b88-b640-ff2e805ca801 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:06.519414 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87086608-7fe6-4dc3-90c9-32d96546ce6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:06.520886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87086608-7fe6-4dc3-90c9-32d96546ce6a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:06.556849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-87086608-7fe6-4dc3-90c9-32d96546ce6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:06.556849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:06.556849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:06.556849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:06.556849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:06.556849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:06.556849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:06.556849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:06.556849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:06.556849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:06.556849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:06.556849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:06.556849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:06.556849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:06.556849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:06.557833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:06.557833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:06.557833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:06.557833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:06.557833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:06.557833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:06.557833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:06.557833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:06.557833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:06.557833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:06.557833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:06.557833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:06.557833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:06.557833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:06.557833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:06.557833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:06.557833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:06.558764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:06.558764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:06.558764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:06.558764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:06.562742 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 299/623] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:06.571542 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87086608-7fe6-4dc3-90c9-32d96546ce6a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:06.573751 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6a9d8-1709-4491-8733-40849f41c6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:06.574423 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6a9d8-1709-4491-8733-40849f41c6f0 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:06.574737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6a9d8-1709-4491-8733-40849f41c6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:06.575012 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6a9d8-1709-4491-8733-40849f41c6f0 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:06.576375 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6a9d8-1709-4491-8733-40849f41c6f0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:06.576686 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6a9d8-1709-4491-8733-40849f41c6f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:06.578116 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6a9d8-1709-4491-8733-40849f41c6f0 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:06.602899 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 325/624] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jun 23 11:26:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:06.604207 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6a9d8-1709-4491-8733-40849f41c6f0 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:06.639126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:06.675347 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:06.719463 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87086608-7fe6-4dc3-90c9-32d96546ce6a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:06.730177 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6a9d8-1709-4491-8733-40849f41c6f0 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:06.754840 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3a548-5be7-4478-8054-cc0783feb059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:06.755501 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3a548-5be7-4478-8054-cc0783feb059 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:06.755780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3a548-5be7-4478-8054-cc0783feb059 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:06.756046 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3a548-5be7-4478-8054-cc0783feb059 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:06.756875 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3a548-5be7-4478-8054-cc0783feb059 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:06.757296 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3a548-5be7-4478-8054-cc0783feb059 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:06.758696 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3a548-5be7-4478-8054-cc0783feb059 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:06.768235 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a64b0-3ce6-4851-b64f-85a93fa77484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:06.768937 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a64b0-3ce6-4851-b64f-85a93fa77484 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:06.769166 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a64b0-3ce6-4851-b64f-85a93fa77484 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:06.769443 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a64b0-3ce6-4851-b64f-85a93fa77484 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:06.772837 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 300/625] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:06.778079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cb183-fc9f-47f4-922b-c805e217185a None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:06.778079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cb183-fc9f-47f4-922b-c805e217185a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:06.778079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cb183-fc9f-47f4-922b-c805e217185a None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:06.778079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 301/626] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:06.778079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a59b5-376b-4a1f-a176-6df4cf1bf6fc None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:06.778079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a59b5-376b-4a1f-a176-6df4cf1bf6fc None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:06.779277 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a59b5-376b-4a1f-a176-6df4cf1bf6fc None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:06.781116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 302/627] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:06.781116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a59b5-376b-4a1f-a176-6df4cf1bf6fc None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:06.855047 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a64b0-3ce6-4851-b64f-85a93fa77484 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:06.857676 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a64b0-3ce6-4851-b64f-85a93fa77484 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:06.890182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a59b5-376b-4a1f-a176-6df4cf1bf6fc None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:06.891600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:06.899985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:06.926637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448bec6d-728e-40a4-b854-18b81f35ccd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:06.927435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bec6d-728e-40a4-b854-18b81f35ccd9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:06.927718 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bec6d-728e-40a4-b854-18b81f35ccd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:06.927990 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bec6d-728e-40a4-b854-18b81f35ccd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:06.935181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448bec6d-728e-40a4-b854-18b81f35ccd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:06.936712 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448bec6d-728e-40a4-b854-18b81f35ccd9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:06.952041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-448bec6d-728e-40a4-b854-18b81f35ccd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:06.952041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:06.952041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:06.952041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:06.952041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:06.952041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:06.952041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:06.952041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:06.952041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:06.952041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:06.952041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:06.952041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:06.952041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:06.952041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:06.952041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:06.952965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:06.952965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:06.952965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:06.952965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:06.952965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:06.952965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:06.952965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:06.952965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:06.952965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:06.952965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:06.952965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:06.952965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:06.952965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:06.952965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:06.952965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:06.952965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:06.952965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:06.953923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:06.953923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:06.953923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:06.953923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:06.953923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:06.953923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:06.953923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:06.953923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:06.953923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 303/628] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:06.954495 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4d606-7bf7-4640-b2af-9afafa836320 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:06.955096 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4d606-7bf7-4640-b2af-9afafa836320 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:06.955096 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4d606-7bf7-4640-b2af-9afafa836320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:06.975498 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 326/629] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:26:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:26:06.975711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-28e4d606-7bf7-4640-b2af-9afafa836320 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:06.975819 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0d33f-aea9-47b9-842c-87673f67ae10 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:06.975819 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0d33f-aea9-47b9-842c-87673f67ae10 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:06.975979 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-28e4d606-7bf7-4640-b2af-9afafa836320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:06.976066 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0d33f-aea9-47b9-842c-87673f67ae10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:06.992577 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-a8d0d33f-aea9-47b9-842c-87673f67ae10 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:06.993823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a8d0d33f-aea9-47b9-842c-87673f67ae10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:07.042597 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 304/630] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:26:07.042597 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4d606-7bf7-4640-b2af-9afafa836320 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:07.073483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 327/631] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:26:07.074695 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d0d33f-aea9-47b9-842c-87673f67ae10 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:07.141384 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:07.155374 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:07.159125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4d606-7bf7-4640-b2af-9afafa836320 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:07.186324 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d0d33f-aea9-47b9-842c-87673f67ae10 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:07.194737 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aecc685-36e2-403a-b61b-baa2a9bf07ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:07.195441 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aecc685-36e2-403a-b61b-baa2a9bf07ed None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:07.196448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aecc685-36e2-403a-b61b-baa2a9bf07ed None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:07.197071 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aecc685-36e2-403a-b61b-baa2a9bf07ed None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:07.197860 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aecc685-36e2-403a-b61b-baa2a9bf07ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:07.198725 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aecc685-36e2-403a-b61b-baa2a9bf07ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:07.200205 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aecc685-36e2-403a-b61b-baa2a9bf07ed None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:07.216885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 305/632] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:07.216885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aecc685-36e2-403a-b61b-baa2a9bf07ed None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:07.227212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81e930-d73f-44eb-9dfe-5a6b9900c0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:07.228005 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81e930-d73f-44eb-9dfe-5a6b9900c0dd None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:07.228293 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81e930-d73f-44eb-9dfe-5a6b9900c0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:07.228562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81e930-d73f-44eb-9dfe-5a6b9900c0dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:07.241135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81e930-d73f-44eb-9dfe-5a6b9900c0dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:07.242612 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81e930-d73f-44eb-9dfe-5a6b9900c0dd None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:07.260401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-ed81e930-d73f-44eb-9dfe-5a6b9900c0dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:07.260401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:07.260401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:07.260401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:07.260401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:07.260401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:07.260401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:07.260401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:07.260401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:07.260401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:07.260401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:07.260401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:07.260401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:07.260401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:07.260401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:07.261335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:07.261335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:07.261335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:07.261335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:07.261335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:07.261335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:07.261335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:07.261335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:07.261335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:07.261335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:07.261335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:07.261335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:07.261335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:07.261335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:07.261335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:07.261335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:07.261335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:07.263391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:07.263391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:07.263391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:07.263391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:07.263391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 328/633] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:07.263391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81e930-d73f-44eb-9dfe-5a6b9900c0dd None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:07.326652 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aecc685-36e2-403a-b61b-baa2a9bf07ed None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:07.360946 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f261e4-f1c8-4d50-92f2-d56ba90095c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:07.361744 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f261e4-f1c8-4d50-92f2-d56ba90095c3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:07.362026 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f261e4-f1c8-4d50-92f2-d56ba90095c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:07.362300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f261e4-f1c8-4d50-92f2-d56ba90095c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:07.368460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81e930-d73f-44eb-9dfe-5a6b9900c0dd None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:07.372721 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f261e4-f1c8-4d50-92f2-d56ba90095c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:07.380458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-e0f261e4-f1c8-4d50-92f2-d56ba90095c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:07.380458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:07.380458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:07.380458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:07.380458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:07.380458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:07.380458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:07.380458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:07.380458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:07.380458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:07.380458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:07.380458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:07.380458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:07.380458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:07.380458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:07.380458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:07.381391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:07.381391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:07.381391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:07.381391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:07.381391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:07.381391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:07.381391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:07.381391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:07.381391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:07.381391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:07.381391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:07.381391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:07.381391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:07.381391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:07.381391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:07.381391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:07.382886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:07.382886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:07.382886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:07.382886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:07.382886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:07.382886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:07.382886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:07.382886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:07.382886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:07.382886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:07.382886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:07.382886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:07.382886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 306/634] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:26:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:07.382886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f261e4-f1c8-4d50-92f2-d56ba90095c3 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:07.387462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:07.403253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:07.410959 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62397d-83fb-4c4d-8585-565d6f256d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:07.411752 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62397d-83fb-4c4d-8585-565d6f256d76 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:07.412081 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62397d-83fb-4c4d-8585-565d6f256d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:07.412352 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62397d-83fb-4c4d-8585-565d6f256d76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:07.423466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62397d-83fb-4c4d-8585-565d6f256d76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:07.431013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-0d62397d-83fb-4c4d-8585-565d6f256d76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:07.431013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:07.431013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:07.431013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:07.431013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:07.431013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:07.431013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:07.431013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:07.431013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:07.431013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:07.431013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:07.431013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:07.431013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:07.431013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:07.431013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:07.431013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:07.432806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:07.432806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:07.432806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:07.432806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:07.432806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:07.432806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:07.432806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:07.432806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:07.432806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:07.432806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:07.432806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:07.432806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:07.432806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:07.432806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:07.432806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:07.432806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:07.433628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:07.433628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:07.433628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:07.433628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:07.433628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:07.433628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:07.433628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:07.433628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:07.433628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:07.433628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:07.433628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:07.433628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:07.434466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 329/635] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:26:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:07.436022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62397d-83fb-4c4d-8585-565d6f256d76 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:07.490052 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f261e4-f1c8-4d50-92f2-d56ba90095c3 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:07.522367 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a58e83-4c33-4d44-8477-5f06952f6eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:07.523276 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a58e83-4c33-4d44-8477-5f06952f6eef None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:07.523549 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a58e83-4c33-4d44-8477-5f06952f6eef None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:07.523817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a58e83-4c33-4d44-8477-5f06952f6eef None admin] PATH_INFO: `/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/450b1fad96604fb09d0705895c6d3da4/roles/13708a4819a94b94b47ba5994aaa9310` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:07.539125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a58e83-4c33-4d44-8477-5f06952f6eef None admin] RBAC: Authorizing `identity:create_grant(project_id=886c2939a14b48febc2e2fe639c8f1e4, user_id=450b1fad96604fb09d0705895c6d3da4, role_id=13708a4819a94b94b47ba5994aaa9310)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:07.540584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a58e83-4c33-4d44-8477-5f06952f6eef None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:07.540700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62397d-83fb-4c4d-8585-565d6f256d76 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:07.573288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4bdc9-b9ac-4330-9867-e2e26533690d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:07.573932 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4bdc9-b9ac-4330-9867-e2e26533690d None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:07.574211 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4bdc9-b9ac-4330-9867-e2e26533690d None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:07.574483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4bdc9-b9ac-4330-9867-e2e26533690d None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:07.575828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4bdc9-b9ac-4330-9867-e2e26533690d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:07.576135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4bdc9-b9ac-4330-9867-e2e26533690d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:07.577481 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4bdc9-b9ac-4330-9867-e2e26533690d None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:07.588838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 330/636] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:07.589592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4bdc9-b9ac-4330-9867-e2e26533690d None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:07.607980 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 307/637] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:26:07 2022] PUT /identity/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/450b1fad96604fb09d0705895c6d3da4/roles/13708a4819a94b94b47ba5994aaa9310 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:26:07.609270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda1276-3704-4772-a4cf-7df81ff1acc6 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:07.609568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda1276-3704-4772-a4cf-7df81ff1acc6 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:07.609841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda1276-3704-4772-a4cf-7df81ff1acc6 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:07.611163 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 308/638] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:07.611890 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda1276-3704-4772-a4cf-7df81ff1acc6 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:07.621358 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:07.702428 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:07.703764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4bdc9-b9ac-4330-9867-e2e26533690d None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:07.738397 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda1276-3704-4772-a4cf-7df81ff1acc6 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:07.740754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fec41-57c0-4830-ad43-5bb2caae95c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:07.741543 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fec41-57c0-4830-ad43-5bb2caae95c6 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:07.741830 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fec41-57c0-4830-ad43-5bb2caae95c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:07.742107 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fec41-57c0-4830-ad43-5bb2caae95c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:07.742898 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875fec41-57c0-4830-ad43-5bb2caae95c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:07.743210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875fec41-57c0-4830-ad43-5bb2caae95c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:07.744838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875fec41-57c0-4830-ad43-5bb2caae95c6 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:07.754790 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 331/639] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:26:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:07.755781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fec41-57c0-4830-ad43-5bb2caae95c6 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:07.788480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6236c0f-acd8-4cf5-91a8-95112f957166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:07.789264 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6236c0f-acd8-4cf5-91a8-95112f957166 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:07.789562 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6236c0f-acd8-4cf5-91a8-95112f957166 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:07.789830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6236c0f-acd8-4cf5-91a8-95112f957166 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:07.790680 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6236c0f-acd8-4cf5-91a8-95112f957166 None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:07.790995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6236c0f-acd8-4cf5-91a8-95112f957166 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:07.792492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6236c0f-acd8-4cf5-91a8-95112f957166 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:07.801820 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 309/640] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:26:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:07.803257 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c88644-667b-402c-a66d-46fbd26f1642 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:07.803555 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c88644-667b-402c-a66d-46fbd26f1642 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:07.803837 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c88644-667b-402c-a66d-46fbd26f1642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:07.822569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d5c88644-667b-402c-a66d-46fbd26f1642 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:07.823832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d5c88644-667b-402c-a66d-46fbd26f1642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:07.863804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:07.870057 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fec41-57c0-4830-ad43-5bb2caae95c6 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:07.905999 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 310/641] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:26:07.905999 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c88644-667b-402c-a66d-46fbd26f1642 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:07.922880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad5cf3-cae8-4d52-bb11-23e18c6d8dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:07.923745 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad5cf3-cae8-4d52-bb11-23e18c6d8dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:07.924028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad5cf3-cae8-4d52-bb11-23e18c6d8dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:07.924467 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad5cf3-cae8-4d52-bb11-23e18c6d8dc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:07.932130 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad5cf3-cae8-4d52-bb11-23e18c6d8dc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:07.933594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad5cf3-cae8-4d52-bb11-23e18c6d8dc3 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:07.948290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-bfad5cf3-cae8-4d52-bb11-23e18c6d8dc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:07.948290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:07.948290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:07.948290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:07.948290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:07.948290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:07.948290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:07.948290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:07.948290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:07.948290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:07.948290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:07.948290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:07.948290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:07.948290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:07.948290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:07.949208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:07.949208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:07.949208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:07.949208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:07.949208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:07.949208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:07.949208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:07.949208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:07.949208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:07.949208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:07.949208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:07.949208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:07.949208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:07.949208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:07.949208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:07.949208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:07.949208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:07.950098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:07.950098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:07.950098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:07.950098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:07.950098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:07.950098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:07.950098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:07.950098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:07.950098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 332/642] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:07.950666 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad5cf3-cae8-4d52-bb11-23e18c6d8dc3 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:07.966512 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:08.011401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c88644-667b-402c-a66d-46fbd26f1642 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:08.049845 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3322ce0-b707-442d-9f25-5a3af266deae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:08.050782 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3322ce0-b707-442d-9f25-5a3af266deae None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:08.051072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3322ce0-b707-442d-9f25-5a3af266deae None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:08.051349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3322ce0-b707-442d-9f25-5a3af266deae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:08.054425 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad5cf3-cae8-4d52-bb11-23e18c6d8dc3 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:08.062811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3322ce0-b707-442d-9f25-5a3af266deae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:08.064424 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3322ce0-b707-442d-9f25-5a3af266deae None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:08.083697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-a3322ce0-b707-442d-9f25-5a3af266deae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:08.083697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:08.083697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:08.083697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:08.083697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:08.083697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:08.083697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:08.083697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:08.083697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:08.083697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:08.083697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:08.083697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:08.083697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:08.083697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:08.083697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:08.084716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:08.084716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:08.084716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:08.084716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:08.084716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:08.084716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:08.084716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:08.084716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:08.084716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:08.084716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:08.084716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:08.084716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:08.084716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:08.084716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:08.084716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:08.084716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:08.084716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:08.085613 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:08.085613 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:08.085613 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:08.085613 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:08.085613 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 311/643] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:08.085895 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3322ce0-b707-442d-9f25-5a3af266deae None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:08.093402 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0e3da-04e8-47a4-81d8-e726b68e033f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:08.093573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0e3da-04e8-47a4-81d8-e726b68e033f None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:08.093719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0e3da-04e8-47a4-81d8-e726b68e033f None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:08.093995 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0e3da-04e8-47a4-81d8-e726b68e033f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:08.104731 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0e3da-04e8-47a4-81d8-e726b68e033f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:08.106194 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0e3da-04e8-47a4-81d8-e726b68e033f None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:08.124860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-cae0e3da-04e8-47a4-81d8-e726b68e033f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:08.124860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:08.124860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:08.124860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:08.124860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:08.124860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:08.124860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:08.124860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:08.124860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:08.124860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:08.124860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:08.124860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:08.124860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:08.124860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:08.124860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:08.125799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:08.125799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:08.125799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:08.125799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:08.125799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:08.125799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:08.125799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:08.125799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:08.125799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:08.125799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:08.125799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:08.125799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:08.125799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:08.125799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:08.125799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:08.125799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:08.125799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:08.126802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:08.126802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:08.126802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:08.126802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:08.126802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 333/644] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:08.127120 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0e3da-04e8-47a4-81d8-e726b68e033f None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:08.137171 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:08.201849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3322ce0-b707-442d-9f25-5a3af266deae None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:08.215355 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:08.234320 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0e3da-04e8-47a4-81d8-e726b68e033f None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:08.238467 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b33e2-f231-4bde-b401-61094255bd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:08.239306 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b33e2-f231-4bde-b401-61094255bd6c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:08.239592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b33e2-f231-4bde-b401-61094255bd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:08.239869 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b33e2-f231-4bde-b401-61094255bd6c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:08.251308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402b33e2-f231-4bde-b401-61094255bd6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:08.259918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-402b33e2-f231-4bde-b401-61094255bd6c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:08.259918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:08.259918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:08.259918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:08.259918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:08.259918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:08.259918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:08.259918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:08.259918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:08.259918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:08.259918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:08.259918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:08.259918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:08.259918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:08.259918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:08.259918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:08.260854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:08.260854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:08.260854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:08.260854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:08.260854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:08.260854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:08.260854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:08.260854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:08.260854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:08.260854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:08.260854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:08.260854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:08.260854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:08.260854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:08.260854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:08.260854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:08.261682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:08.261682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:08.261682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:08.261682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:08.261682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:08.261682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:08.261682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:08.261682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:08.261682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:08.261682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:08.261682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:08.261682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:08.261682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 312/645] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:26:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:08.262391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b33e2-f231-4bde-b401-61094255bd6c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:08.275465 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd1fae-1c6c-441e-9ce8-0a32554984da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:08.276131 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd1fae-1c6c-441e-9ce8-0a32554984da None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:08.276427 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd1fae-1c6c-441e-9ce8-0a32554984da None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:08.276709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd1fae-1c6c-441e-9ce8-0a32554984da None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:08.277501 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd1fae-1c6c-441e-9ce8-0a32554984da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:08.277813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd1fae-1c6c-441e-9ce8-0a32554984da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:08.280080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd1fae-1c6c-441e-9ce8-0a32554984da None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:08.297642 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 334/646] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:08.297642 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd1fae-1c6c-441e-9ce8-0a32554984da None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:08.364870 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:08.378130 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b33e2-f231-4bde-b401-61094255bd6c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:08.401519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd1fae-1c6c-441e-9ce8-0a32554984da None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:08.418057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d257f0-9dfd-4a8a-b8aa-4478cdc6e943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:08.419147 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d257f0-9dfd-4a8a-b8aa-4478cdc6e943 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:08.419589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d257f0-9dfd-4a8a-b8aa-4478cdc6e943 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:08.419958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d257f0-9dfd-4a8a-b8aa-4478cdc6e943 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:08.421430 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d257f0-9dfd-4a8a-b8aa-4478cdc6e943 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:08.421430 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d257f0-9dfd-4a8a-b8aa-4478cdc6e943 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:08.422968 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d257f0-9dfd-4a8a-b8aa-4478cdc6e943 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:08.436532 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ad0f8-aaa4-45a9-bd06-6b1db43fbb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:08.437186 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ad0f8-aaa4-45a9-bd06-6b1db43fbb1e None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:08.437471 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ad0f8-aaa4-45a9-bd06-6b1db43fbb1e None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:08.437745 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ad0f8-aaa4-45a9-bd06-6b1db43fbb1e None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:08.438461 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ad0f8-aaa4-45a9-bd06-6b1db43fbb1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:08.443625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ad0f8-aaa4-45a9-bd06-6b1db43fbb1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:08.443750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 313/647] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:08.443750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d257f0-9dfd-4a8a-b8aa-4478cdc6e943 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:08.443907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ad0f8-aaa4-45a9-bd06-6b1db43fbb1e None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:08.453146 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 335/648] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:08.457500 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ad0f8-aaa4-45a9-bd06-6b1db43fbb1e None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:08.491423 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:08.563875 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d257f0-9dfd-4a8a-b8aa-4478cdc6e943 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:08.565266 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ad0f8-aaa4-45a9-bd06-6b1db43fbb1e None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:08.595703 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:08.600112 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fdb80-aa57-4320-a574-0338b57b1da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:08.600908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fdb80-aa57-4320-a574-0338b57b1da2 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:08.601260 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fdb80-aa57-4320-a574-0338b57b1da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:08.601601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fdb80-aa57-4320-a574-0338b57b1da2 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:08.602417 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28fdb80-aa57-4320-a574-0338b57b1da2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:08.602819 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28fdb80-aa57-4320-a574-0338b57b1da2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:08.604327 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28fdb80-aa57-4320-a574-0338b57b1da2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:08.604701 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e732a2-e300-4ff9-92f1-410d76c75c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:08.605408 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e732a2-e300-4ff9-92f1-410d76c75c97 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:08.605910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e732a2-e300-4ff9-92f1-410d76c75c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:08.606247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e732a2-e300-4ff9-92f1-410d76c75c97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:08.607498 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e732a2-e300-4ff9-92f1-410d76c75c97 None admin] RBAC: Adding query filter params (role.id=13708a4819a94b94b47ba5994aaa9310, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=161c5ed1e47e450290a1f428ee100b68) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:08.607875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e732a2-e300-4ff9-92f1-410d76c75c97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:08.609560 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e732a2-e300-4ff9-92f1-410d76c75c97 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:08.612776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 314/649] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:26:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:08.613560 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fdb80-aa57-4320-a574-0338b57b1da2 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:08.627108 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 336/650] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:26:08 2022] GET /identity/v3/role_assignments?user.id=161c5ed1e47e450290a1f428ee100b68&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=13708a4819a94b94b47ba5994aaa9310 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:08.627108 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e732a2-e300-4ff9-92f1-410d76c75c97 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:08.724542 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:08.732757 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fdb80-aa57-4320-a574-0338b57b1da2 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:08.768390 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e732a2-e300-4ff9-92f1-410d76c75c97 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:08.772932 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecadd3b-772e-4c07-aa8e-bfd661f8ad29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:08.773895 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecadd3b-772e-4c07-aa8e-bfd661f8ad29 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:08.774279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecadd3b-772e-4c07-aa8e-bfd661f8ad29 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:08.774970 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecadd3b-772e-4c07-aa8e-bfd661f8ad29 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:08.788070 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecadd3b-772e-4c07-aa8e-bfd661f8ad29 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:08.790142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecadd3b-772e-4c07-aa8e-bfd661f8ad29 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:08.807568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-eecadd3b-772e-4c07-aa8e-bfd661f8ad29 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 11:26:08.807568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:08.807568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:08.807568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:08.807568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:08.807568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:08.807568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:08.807568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:08.807568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:08.807568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:08.807568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:08.807568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:08.807568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:08.807568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:08.807568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:08.807568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:08.808905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:08.808905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:08.808905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:08.808905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:08.808905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:08.808905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:08.808905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:08.808905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:08.808905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:08.808905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:08.808905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:08.808905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:08.808905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:08.808905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:08.808905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:08.808905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:08.808905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:08.808905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:08.810702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:08.810702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:08.810702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:08.810702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:08.810702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:08.810702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:08.810702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:08.810702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:08.810702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 11:26:08.810702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:08.813522 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 315/651] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:26:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:08.814848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecadd3b-772e-4c07-aa8e-bfd661f8ad29 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:08.820425 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a92b7-451c-4b73-b509-8e62a64e4696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:08.821236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a92b7-451c-4b73-b509-8e62a64e4696 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:08.821622 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a92b7-451c-4b73-b509-8e62a64e4696 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:08.821926 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a92b7-451c-4b73-b509-8e62a64e4696 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:08.835758 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a92b7-451c-4b73-b509-8e62a64e4696 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:08.837462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a92b7-451c-4b73-b509-8e62a64e4696 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:08.857574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-0b8a92b7-451c-4b73-b509-8e62a64e4696 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 11:26:08.857574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:08.857574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:08.857574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:08.857574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:08.857574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:08.857574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:08.857574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:08.857574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:08.857574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:08.857574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:08.857574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:08.857574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:08.857574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:08.857574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:08.857574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:08.858595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:08.858595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:08.858595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:08.858595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:08.858595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:08.858595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:08.858595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:08.858595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:08.858595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:08.858595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:08.858595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:08.858595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:08.858595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:08.858595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:08.858595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:08.858595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:08.858595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:08.858595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:08.859557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:08.859557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:08.859557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:08.859557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:08.859557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:08.859557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:08.859557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:08.859557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:08.859557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 11:26:08.859557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:08.859557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 337/652] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:26:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:08.860206 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a92b7-451c-4b73-b509-8e62a64e4696 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:08.923026 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:08.945199 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecadd3b-772e-4c07-aa8e-bfd661f8ad29 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:08.978843 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a92b7-451c-4b73-b509-8e62a64e4696 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:08.982223 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494673c-b1ec-45fa-8132-0cc49b66e095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:08.983054 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494673c-b1ec-45fa-8132-0cc49b66e095 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:08.983338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494673c-b1ec-45fa-8132-0cc49b66e095 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:08.983612 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494673c-b1ec-45fa-8132-0cc49b66e095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:08.994453 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494673c-b1ec-45fa-8132-0cc49b66e095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:08.996038 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494673c-b1ec-45fa-8132-0cc49b66e095 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:09.014217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-b494673c-b1ec-45fa-8132-0cc49b66e095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:09.014217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:09.014217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:09.014217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:09.014217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:09.014217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:09.014217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:09.014217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:09.014217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:09.014217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:09.014217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:09.014217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:09.014217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:09.014217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:09.014217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:09.015258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:09.015258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:09.015258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:09.015258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:09.015258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:09.015258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:09.015258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:09.015258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:09.015258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:09.015258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:09.015258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:09.015258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:09.015258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:09.015258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:09.015258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:09.015258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:09.015258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:09.016150 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:09.016150 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:09.016150 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:09.016150 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:09.016150 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 316/653] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:09.016621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494673c-b1ec-45fa-8132-0cc49b66e095 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:09.021518 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5403825-4e76-400b-98b2-bbc57c51b478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:09.022425 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5403825-4e76-400b-98b2-bbc57c51b478 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:09.022909 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5403825-4e76-400b-98b2-bbc57c51b478 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:09.023360 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5403825-4e76-400b-98b2-bbc57c51b478 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:09.024389 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5403825-4e76-400b-98b2-bbc57c51b478 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:09.024899 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5403825-4e76-400b-98b2-bbc57c51b478 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:09.026797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:09.028147 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5403825-4e76-400b-98b2-bbc57c51b478 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:09.052724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 338/654] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Jun 23 11:26:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:09.053859 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5403825-4e76-400b-98b2-bbc57c51b478 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:09.133439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494673c-b1ec-45fa-8132-0cc49b66e095 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:09.175022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5403825-4e76-400b-98b2-bbc57c51b478 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:09.181688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3dfd3a-21d5-48bb-b3cf-45784cc736d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:09.182637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3dfd3a-21d5-48bb-b3cf-45784cc736d5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:09.183683 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3dfd3a-21d5-48bb-b3cf-45784cc736d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:09.184165 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3dfd3a-21d5-48bb-b3cf-45784cc736d5 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:09.186501 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:09.187283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3dfd3a-21d5-48bb-b3cf-45784cc736d5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:09.188780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3dfd3a-21d5-48bb-b3cf-45784cc736d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:09.190938 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3dfd3a-21d5-48bb-b3cf-45784cc736d5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:09.214866 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 317/655] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Jun 23 11:26:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:09.220353 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3dfd3a-21d5-48bb-b3cf-45784cc736d5 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:09.223432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930aea5-8676-4cbf-b189-de1bf600c8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:09.225233 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930aea5-8676-4cbf-b189-de1bf600c8eb None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:09.225710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930aea5-8676-4cbf-b189-de1bf600c8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:09.226182 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930aea5-8676-4cbf-b189-de1bf600c8eb None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:09.227131 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c930aea5-8676-4cbf-b189-de1bf600c8eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:09.227615 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c930aea5-8676-4cbf-b189-de1bf600c8eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:09.229167 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c930aea5-8676-4cbf-b189-de1bf600c8eb None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:09.245877 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 339/656] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:09.247894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930aea5-8676-4cbf-b189-de1bf600c8eb None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:09.302621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:09.356002 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3dfd3a-21d5-48bb-b3cf-45784cc736d5 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:09.370740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930aea5-8676-4cbf-b189-de1bf600c8eb None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:09.398258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9bb2c-d606-4f55-84f2-18c38b9e040a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:09.399033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9bb2c-d606-4f55-84f2-18c38b9e040a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:09.399402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9bb2c-d606-4f55-84f2-18c38b9e040a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:09.399750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9bb2c-d606-4f55-84f2-18c38b9e040a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:09.417067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79d14b-d00f-4573-a1b3-9728200a35c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:09.418522 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79d14b-d00f-4573-a1b3-9728200a35c1 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:09.419594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79d14b-d00f-4573-a1b3-9728200a35c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:09.420281 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79d14b-d00f-4573-a1b3-9728200a35c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:09.469027 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:09.476700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9bb2c-d606-4f55-84f2-18c38b9e040a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:09.478417 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9bb2c-d606-4f55-84f2-18c38b9e040a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:09.511895 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79d14b-d00f-4573-a1b3-9728200a35c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:09.514388 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79d14b-d00f-4573-a1b3-9728200a35c1 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:09.542254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:09.588524 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 318/657] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:26:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:26:09.589573 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9bb2c-d606-4f55-84f2-18c38b9e040a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:09.629293 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 340/658] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:26:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:26:09.630386 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79d14b-d00f-4573-a1b3-9728200a35c1 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:09.701028 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9bb2c-d606-4f55-84f2-18c38b9e040a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:09.702826 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:09.738022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79d14b-d00f-4573-a1b3-9728200a35c1 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:09.742272 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7c1d7-d69f-43d7-96a2-cea38e275aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:09.743097 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7c1d7-d69f-43d7-96a2-cea38e275aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:09.743381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7c1d7-d69f-43d7-96a2-cea38e275aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:09.743653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7c1d7-d69f-43d7-96a2-cea38e275aa3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:09.753110 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7c1d7-d69f-43d7-96a2-cea38e275aa3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:09.754573 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7c1d7-d69f-43d7-96a2-cea38e275aa3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:09.769284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-44f7c1d7-d69f-43d7-96a2-cea38e275aa3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 11:26:09.769284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:09.769284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:09.769284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:09.769284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:09.769284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:09.769284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:09.769284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:09.769284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:09.769284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:09.769284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:09.769284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:09.769284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:09.769284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:09.769284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:09.769284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:09.770288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:09.770288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:09.770288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:09.770288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:09.770288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:09.770288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:09.770288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:09.770288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:09.770288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:09.770288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:09.770288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:09.770288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:09.770288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:09.770288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:09.770288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:09.770288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:09.770288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:09.770288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:09.771279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:09.771279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:09.771279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:09.771279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:09.771279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:09.771279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:09.771279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:09.771279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:09.771279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 11:26:09.771279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:09.771279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 319/659] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:26:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:09.771279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7c1d7-d69f-43d7-96a2-cea38e275aa3 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:09.780110 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bd213-440f-4f47-ab6d-93dda192baeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:09.780912 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bd213-440f-4f47-ab6d-93dda192baeb None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:09.781198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bd213-440f-4f47-ab6d-93dda192baeb None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:09.781475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bd213-440f-4f47-ab6d-93dda192baeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:09.792970 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38bd213-440f-4f47-ab6d-93dda192baeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:09.795404 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38bd213-440f-4f47-ab6d-93dda192baeb None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:09.798594 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:09.813228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-d38bd213-440f-4f47-ab6d-93dda192baeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:09.813228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:09.813228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:09.813228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:09.813228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:09.813228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:09.813228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:09.813228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:09.813228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:09.813228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:09.813228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:09.813228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:09.813228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:09.813228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:09.813228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:09.814214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:09.814214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:09.814214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:09.814214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:09.814214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:09.814214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:09.814214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:09.814214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:09.814214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:09.814214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:09.814214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:09.814214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:09.814214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:09.814214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:09.814214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:09.814214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:09.814214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:09.815167 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:09.815167 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:09.815167 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:09.815167 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:09.815167 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 341/660] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:09.816048 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ab3c6-aab9-4489-a2aa-d971071daec8 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:09.816403 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ab3c6-aab9-4489-a2aa-d971071daec8 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:09.816677 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ab3c6-aab9-4489-a2aa-d971071daec8 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:09.820328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 342/661] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:09.820328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ab3c6-aab9-4489-a2aa-d971071daec8 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:09.883422 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7c1d7-d69f-43d7-96a2-cea38e275aa3 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:09.921756 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9966b-ab9e-4655-ae3d-9dfb1b16e25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:09.923318 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9966b-ab9e-4655-ae3d-9dfb1b16e25d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:09.923713 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9966b-ab9e-4655-ae3d-9dfb1b16e25d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:09.923994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9966b-ab9e-4655-ae3d-9dfb1b16e25d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:09.928672 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ab3c6-aab9-4489-a2aa-d971071daec8 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:09.937502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a9966b-ab9e-4655-ae3d-9dfb1b16e25d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:09.939636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a9966b-ab9e-4655-ae3d-9dfb1b16e25d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:09.952968 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:09.963948 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-75a9966b-ab9e-4655-ae3d-9dfb1b16e25d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:09.963948 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:09.963948 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:09.963948 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:09.963948 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:09.963948 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:09.963948 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:09.963948 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:09.963948 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:09.963948 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:09.963948 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:09.963948 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:09.963948 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:09.963948 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:09.963948 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:09.964904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:09.964904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:09.964904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:09.964904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:09.964904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:09.964904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:09.964904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:09.964904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:09.964904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:09.964904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:09.964904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:09.964904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:09.964904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:09.964904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:09.964904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:09.964904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:09.964904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:09.967021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:09.967021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:09.967021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:09.967021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:09.967021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 320/662] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:09.967021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9966b-ab9e-4655-ae3d-9dfb1b16e25d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:09.974301 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444f94e-2c7a-43dc-a6ae-fd3da6cb10f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:09.975698 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444f94e-2c7a-43dc-a6ae-fd3da6cb10f2 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:09.975992 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444f94e-2c7a-43dc-a6ae-fd3da6cb10f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:09.976269 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444f94e-2c7a-43dc-a6ae-fd3da6cb10f2 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:09.977138 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444f94e-2c7a-43dc-a6ae-fd3da6cb10f2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:09.977460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444f94e-2c7a-43dc-a6ae-fd3da6cb10f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:09.978982 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444f94e-2c7a-43dc-a6ae-fd3da6cb10f2 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:09.999458 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 343/663] 10.222.0.13 () {56 vars in 1139 bytes} [Thu Jun 23 11:26:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:10.000841 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0617ac0-fae4-441c-8f8e-8defa81848cd None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:10.001203 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0617ac0-fae4-441c-8f8e-8defa81848cd None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:10.001570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0617ac0-fae4-441c-8f8e-8defa81848cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:10.019347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-a0617ac0-fae4-441c-8f8e-8defa81848cd None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:10.020531 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a0617ac0-fae4-441c-8f8e-8defa81848cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:10.057915 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:10.082389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9966b-ab9e-4655-ae3d-9dfb1b16e25d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:10.093421 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 344/664] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:26:10.095674 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0617ac0-fae4-441c-8f8e-8defa81848cd None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:10.119473 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d50f4-3ea3-49b1-8051-1c80d3ac3447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:10.120323 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d50f4-3ea3-49b1-8051-1c80d3ac3447 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:10.120443 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d50f4-3ea3-49b1-8051-1c80d3ac3447 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:10.120673 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d50f4-3ea3-49b1-8051-1c80d3ac3447 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:10.121418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d50f4-3ea3-49b1-8051-1c80d3ac3447 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:10.121720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d50f4-3ea3-49b1-8051-1c80d3ac3447 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:10.123191 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d50f4-3ea3-49b1-8051-1c80d3ac3447 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:10.134464 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 321/665] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:10.135220 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d50f4-3ea3-49b1-8051-1c80d3ac3447 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:10.219669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0617ac0-fae4-441c-8f8e-8defa81848cd None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:10.229437 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:10.238971 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d50f4-3ea3-49b1-8051-1c80d3ac3447 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:10.254470 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8386b-7e72-4e3f-bbf3-62f7f8a9700b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:10.255167 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8386b-7e72-4e3f-bbf3-62f7f8a9700b None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:10.255457 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8386b-7e72-4e3f-bbf3-62f7f8a9700b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:10.255734 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8386b-7e72-4e3f-bbf3-62f7f8a9700b None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:10.256456 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8386b-7e72-4e3f-bbf3-62f7f8a9700b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:10.256768 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8386b-7e72-4e3f-bbf3-62f7f8a9700b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:10.258174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8386b-7e72-4e3f-bbf3-62f7f8a9700b None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:10.274211 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 345/666] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:10.274807 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb6dcf-7029-4fc9-b5be-d29ca3603156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:10.275097 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8386b-7e72-4e3f-bbf3-62f7f8a9700b None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:10.275476 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb6dcf-7029-4fc9-b5be-d29ca3603156 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:10.275760 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb6dcf-7029-4fc9-b5be-d29ca3603156 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:10.276148 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb6dcf-7029-4fc9-b5be-d29ca3603156 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:10.281688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:10.349461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb6dcf-7029-4fc9-b5be-d29ca3603156 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:10.351208 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb6dcf-7029-4fc9-b5be-d29ca3603156 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:10.380843 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8386b-7e72-4e3f-bbf3-62f7f8a9700b None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:10.421379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335154b-70c1-4246-b098-9bc7064f9a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:10.422218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335154b-70c1-4246-b098-9bc7064f9a1a None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:10.422598 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335154b-70c1-4246-b098-9bc7064f9a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:10.422944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335154b-70c1-4246-b098-9bc7064f9a1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:10.433890 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335154b-70c1-4246-b098-9bc7064f9a1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:10.441910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-5335154b-70c1-4246-b098-9bc7064f9a1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:10.441910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:10.441910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:10.441910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:10.441910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:10.441910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:10.441910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:10.441910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:10.441910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:10.441910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:10.441910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:10.441910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:10.441910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:10.441910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:10.441910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:10.441910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:10.443328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:10.443328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:10.443328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:10.443328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:10.443328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:10.443328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:10.443328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:10.443328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:10.443328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:10.443328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:10.443328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:10.443328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:10.443328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:10.443328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:10.443328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:10.443328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:10.444405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:10.444405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:10.444405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:10.444405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:10.444405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:10.444405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:10.444405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:10.444405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:10.444405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:10.444405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:10.444405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:10.444405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:10.450808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 346/667] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:26:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:10.450808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335154b-70c1-4246-b098-9bc7064f9a1a None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:10.467433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 322/668] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:26:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:26:10.468533 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb6dcf-7029-4fc9-b5be-d29ca3603156 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:10.493288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:10.507946 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:10.570461 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335154b-70c1-4246-b098-9bc7064f9a1a None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:10.574401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb6dcf-7029-4fc9-b5be-d29ca3603156 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:10.607261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1027a-8133-4cbf-8dde-b22e24ffe48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:10.608294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1027a-8133-4cbf-8dde-b22e24ffe48a None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:10.608725 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1027a-8133-4cbf-8dde-b22e24ffe48a None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:10.609148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1027a-8133-4cbf-8dde-b22e24ffe48a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:10.609331 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2747d-7857-4b54-9c75-ead3acd01521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:10.610124 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2747d-7857-4b54-9c75-ead3acd01521 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:10.610411 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2747d-7857-4b54-9c75-ead3acd01521 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:10.610719 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2747d-7857-4b54-9c75-ead3acd01521 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:10.616973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1027a-8133-4cbf-8dde-b22e24ffe48a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:10.617561 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d2747d-7857-4b54-9c75-ead3acd01521 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:10.618822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1027a-8133-4cbf-8dde-b22e24ffe48a None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:10.619032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d2747d-7857-4b54-9c75-ead3acd01521 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:10.633737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-13e1027a-8133-4cbf-8dde-b22e24ffe48a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:10.633737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:10.633737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:10.633737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:10.633737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:10.633737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:10.633737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:10.633737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:10.633737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:10.633737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:10.633737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:10.633737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:10.633737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:10.633737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:10.633737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:10.634910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:10.634910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:10.634910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:10.634910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:10.634910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:10.634910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:10.634910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:10.634910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:10.634910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:10.634910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:10.634910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:10.634910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:10.634910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:10.634910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:10.634910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:10.634910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:10.634910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:10.641700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:10.641700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:10.641700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:10.641700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:10.641700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:10.641700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:10.641700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:10.641700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:10.642233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-d8d2747d-7857-4b54-9c75-ead3acd01521 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:10.642233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:10.642233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:10.642233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:10.642233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:10.642233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:10.642233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:10.642233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:10.642233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:10.642233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:10.642233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:10.642233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:10.642233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:10.642233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:10.642233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:10.643515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:10.643515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:10.643515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:10.643515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:10.643515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:10.643515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:10.643515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:10.643515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:10.643515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:10.643515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:10.643515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:10.643515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:10.643515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:10.643515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:10.643515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:10.643515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:10.643515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:10.645523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:10.645523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:10.645523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:10.645523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:10.645523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:10.645523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:10.645523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:10.645523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:10.646052 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 347/669] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:10.646052 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1027a-8133-4cbf-8dde-b22e24ffe48a None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:10.646198 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 323/670] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:10.646198 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2747d-7857-4b54-9c75-ead3acd01521 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:10.738386 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:10.747589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:10.751430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1027a-8133-4cbf-8dde-b22e24ffe48a None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:10.753613 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2747d-7857-4b54-9c75-ead3acd01521 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:10.786133 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651828ef-e7b7-407e-9f95-25a0398c7785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:10.787137 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651828ef-e7b7-407e-9f95-25a0398c7785 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:10.787612 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651828ef-e7b7-407e-9f95-25a0398c7785 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:10.788059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651828ef-e7b7-407e-9f95-25a0398c7785 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:10.789172 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651828ef-e7b7-407e-9f95-25a0398c7785 None admin] RBAC: Adding query filter params (name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:10.789546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651828ef-e7b7-407e-9f95-25a0398c7785 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:10.789952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753a78c-7d67-47a7-b919-391f23ba1993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:10.790779 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753a78c-7d67-47a7-b919-391f23ba1993 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:10.791062 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753a78c-7d67-47a7-b919-391f23ba1993 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:10.791194 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651828ef-e7b7-407e-9f95-25a0398c7785 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:10.791334 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753a78c-7d67-47a7-b919-391f23ba1993 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:10.804472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 348/671] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:26:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:10.804472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651828ef-e7b7-407e-9f95-25a0398c7785 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:10.810618 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753a78c-7d67-47a7-b919-391f23ba1993 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:10.812561 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753a78c-7d67-47a7-b919-391f23ba1993 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:10.836578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-9753a78c-7d67-47a7-b919-391f23ba1993 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:10.836578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:10.836578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:10.836578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:10.836578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:10.836578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:10.836578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:10.836578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:10.836578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:10.836578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:10.836578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:10.836578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:10.836578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:10.836578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:10.836578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:10.837664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:10.837664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:10.837664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:10.837664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:10.837664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:10.837664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:10.837664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:10.837664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:10.837664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:10.837664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:10.837664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:10.837664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:10.837664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:10.837664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:10.837664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:10.837664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:10.837664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:10.839137 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:10.839137 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:10.839137 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:10.839137 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:10.843380 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 324/672] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:10.844253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753a78c-7d67-47a7-b919-391f23ba1993 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:10.912123 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651828ef-e7b7-407e-9f95-25a0398c7785 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:10.947776 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a1411-e3d1-4439-bdcf-636941ebcb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:10.948435 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a1411-e3d1-4439-bdcf-636941ebcb6c None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:10.948715 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a1411-e3d1-4439-bdcf-636941ebcb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:10.949022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a1411-e3d1-4439-bdcf-636941ebcb6c None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:10.949828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a1411-e3d1-4439-bdcf-636941ebcb6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:10.950809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a1411-e3d1-4439-bdcf-636941ebcb6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:10.952350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a1411-e3d1-4439-bdcf-636941ebcb6c None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:10.955506 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753a78c-7d67-47a7-b919-391f23ba1993 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:10.969865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 349/673] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:10.969865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a1411-e3d1-4439-bdcf-636941ebcb6c None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:10.979383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:10.991566 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f8c03-ec05-43d5-b3bc-c942602857a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:10.991566 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f8c03-ec05-43d5-b3bc-c942602857a9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:10.991566 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f8c03-ec05-43d5-b3bc-c942602857a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:10.991878 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f8c03-ec05-43d5-b3bc-c942602857a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:10.992470 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f8c03-ec05-43d5-b3bc-c942602857a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:10.992774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f8c03-ec05-43d5-b3bc-c942602857a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:10.994205 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f8c03-ec05-43d5-b3bc-c942602857a9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:11.005969 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 325/674] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:11.006846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f8c03-ec05-43d5-b3bc-c942602857a9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:11.018848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:11.070552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a1411-e3d1-4439-bdcf-636941ebcb6c None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:11.103526 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fa4f7-4d95-4c78-b7a1-57cfaa638059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:11.104306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fa4f7-4d95-4c78-b7a1-57cfaa638059 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:11.104587 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fa4f7-4d95-4c78-b7a1-57cfaa638059 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:11.104861 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fa4f7-4d95-4c78-b7a1-57cfaa638059 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:11.109640 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f8c03-ec05-43d5-b3bc-c942602857a9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:11.114559 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2fa4f7-4d95-4c78-b7a1-57cfaa638059 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:11.116020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2fa4f7-4d95-4c78-b7a1-57cfaa638059 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:11.132855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-6b2fa4f7-4d95-4c78-b7a1-57cfaa638059 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:11.132855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:11.132855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:11.132855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:11.132855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:11.132855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:11.132855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:11.132855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:11.132855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:11.132855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:11.132855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:11.132855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:11.132855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:11.132855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:11.132855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:11.133922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:11.133922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:11.133922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:11.133922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:11.133922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:11.133922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:11.133922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:11.133922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:11.133922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:11.133922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:11.133922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:11.133922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:11.133922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:11.133922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:11.133922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:11.133922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:11.133922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:11.135073 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:11.135073 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:11.135073 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:11.135073 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:11.135073 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 350/675] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:11.135412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61238089-c33e-4dc7-9397-112c54433d84 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:11.135501 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61238089-c33e-4dc7-9397-112c54433d84 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:11.135736 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61238089-c33e-4dc7-9397-112c54433d84 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:11.137233 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 351/676] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:11.138560 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61238089-c33e-4dc7-9397-112c54433d84 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:11.147802 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a603c-5b8d-4730-938e-895128a68707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:11.148470 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a603c-5b8d-4730-938e-895128a68707 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:11.148756 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a603c-5b8d-4730-938e-895128a68707 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:11.149029 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a603c-5b8d-4730-938e-895128a68707 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:11.149782 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a603c-5b8d-4730-938e-895128a68707 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:11.150090 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a603c-5b8d-4730-938e-895128a68707 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:11.151488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a603c-5b8d-4730-938e-895128a68707 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:11.163796 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 326/677] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:11.164664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a603c-5b8d-4730-938e-895128a68707 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:11.203215 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:11.230583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:11.242165 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61238089-c33e-4dc7-9397-112c54433d84 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:11.265188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a603c-5b8d-4730-938e-895128a68707 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:11.275776 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45daa1ca-ad92-4a9a-969b-62146d98cdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:11.276420 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45daa1ca-ad92-4a9a-969b-62146d98cdff None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:11.276705 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45daa1ca-ad92-4a9a-969b-62146d98cdff None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:11.276988 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45daa1ca-ad92-4a9a-969b-62146d98cdff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:11.277810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45daa1ca-ad92-4a9a-969b-62146d98cdff None admin] RBAC: Adding query filter params (role.id=13708a4819a94b94b47ba5994aaa9310, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=b6105838cbf1409587ee7d9e3be554f8) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:11.278139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45daa1ca-ad92-4a9a-969b-62146d98cdff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:11.279606 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45daa1ca-ad92-4a9a-969b-62146d98cdff None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:11.285259 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 352/678] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:26:11 2022] GET /identity/v3/role_assignments?user.id=b6105838cbf1409587ee7d9e3be554f8&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=13708a4819a94b94b47ba5994aaa9310 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:11.286113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45daa1ca-ad92-4a9a-969b-62146d98cdff None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:11.302184 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ea937-7d21-4ecc-aed9-32d0be279682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:11.303367 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ea937-7d21-4ecc-aed9-32d0be279682 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:11.303828 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ea937-7d21-4ecc-aed9-32d0be279682 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:11.304277 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ea937-7d21-4ecc-aed9-32d0be279682 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:11.305308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ea937-7d21-4ecc-aed9-32d0be279682 None admin] RBAC: Adding query filter params (role.id=13708a4819a94b94b47ba5994aaa9310, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=871dd684032c49d09608f0af05b7a765) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:11.305908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ea937-7d21-4ecc-aed9-32d0be279682 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:11.307733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ea937-7d21-4ecc-aed9-32d0be279682 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:11.316523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 327/679] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:26:11 2022] GET /identity/v3/role_assignments?user.id=871dd684032c49d09608f0af05b7a765&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=13708a4819a94b94b47ba5994aaa9310 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:11.318107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c6415-c034-43ac-be05-8bf47062b95b None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:11.318394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c6415-c034-43ac-be05-8bf47062b95b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:11.318695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c6415-c034-43ac-be05-8bf47062b95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:11.350570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-247c6415-c034-43ac-be05-8bf47062b95b None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:11.352285 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-247c6415-c034-43ac-be05-8bf47062b95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:11.446559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 328/680] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:26:11.446559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c6415-c034-43ac-be05-8bf47062b95b None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:11.466303 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45daa1ca-ad92-4a9a-969b-62146d98cdff None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:11.505905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:11.516894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5153e-ac97-4e04-9632-27fb61945baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:11.517556 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5153e-ac97-4e04-9632-27fb61945baf None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:11.517836 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5153e-ac97-4e04-9632-27fb61945baf None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:11.518107 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5153e-ac97-4e04-9632-27fb61945baf None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:11.518880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5153e-ac97-4e04-9632-27fb61945baf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:11.519183 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5153e-ac97-4e04-9632-27fb61945baf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:11.520560 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5153e-ac97-4e04-9632-27fb61945baf None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:11.528992 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:11.533599 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 353/681] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:11.535097 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5153e-ac97-4e04-9632-27fb61945baf None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:11.580085 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c6415-c034-43ac-be05-8bf47062b95b None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:11.621034 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc7b27-3ae1-4983-a16f-d2cb34caa0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:11.621899 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc7b27-3ae1-4983-a16f-d2cb34caa0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:11.622182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc7b27-3ae1-4983-a16f-d2cb34caa0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:11.622449 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc7b27-3ae1-4983-a16f-d2cb34caa0d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:11.630667 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc7b27-3ae1-4983-a16f-d2cb34caa0d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:11.632404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc7b27-3ae1-4983-a16f-d2cb34caa0d7 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:11.647063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-6dfc7b27-3ae1-4983-a16f-d2cb34caa0d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:11.647063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:11.647063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:11.647063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:11.647063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:11.647063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:11.647063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:11.647063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:11.647063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:11.647063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:11.647063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:11.647063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:11.647063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:11.647063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:11.647063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:11.648009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:11.648009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:11.648009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:11.648009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:11.648009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:11.648009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:11.648009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:11.648009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:11.648009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:11.648009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:11.648009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:11.648009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:11.648009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:11.648009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:11.648009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:11.648009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:11.648009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:11.649350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:11.649350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:11.649350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:11.649350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:11.649350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:11.649350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:11.649350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:11.649350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:11.649350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 329/682] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:11.649834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc7b27-3ae1-4983-a16f-d2cb34caa0d7 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:11.664827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5153e-ac97-4e04-9632-27fb61945baf None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:11.712162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30988f63-843e-4f18-9a03-880885c542b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:11.713356 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30988f63-843e-4f18-9a03-880885c542b0 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:11.713845 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30988f63-843e-4f18-9a03-880885c542b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:11.714800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30988f63-843e-4f18-9a03-880885c542b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:11.728867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30988f63-843e-4f18-9a03-880885c542b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:11.736690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-30988f63-843e-4f18-9a03-880885c542b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:26:11.736690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:11.736690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:11.736690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:11.736690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:11.736690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:11.736690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:11.736690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:11.736690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:11.736690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:11.736690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:11.736690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:11.736690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:11.736690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:11.736690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:11.736690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:11.737750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:11.737750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:11.737750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:11.737750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:11.737750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:11.737750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:11.737750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:11.737750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:11.737750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:11.737750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:11.737750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:11.737750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:11.737750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:11.737750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:11.737750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:11.737750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:11.739288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:11.739288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:11.739288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:11.739288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:11.739288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:11.739288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:11.739288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:11.739288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:11.739288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:11.739288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:11.739288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:26:11.739288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:11.739288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 354/683] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:26:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:11.739288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30988f63-843e-4f18-9a03-880885c542b0 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:11.785307 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc7b27-3ae1-4983-a16f-d2cb34caa0d7 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:11.792946 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:11.813853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:11.826902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572afb9-4ae3-42d8-bcc4-1827361b23ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:11.831517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572afb9-4ae3-42d8-bcc4-1827361b23ed None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:11.831517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572afb9-4ae3-42d8-bcc4-1827361b23ed None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:11.831517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572afb9-4ae3-42d8-bcc4-1827361b23ed None admin] PATH_INFO: `/v3/users/swift` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:11.837157 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6572afb9-4ae3-42d8-bcc4-1827361b23ed None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:11.838746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6572afb9-4ae3-42d8-bcc4-1827361b23ed None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:11.855288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-6572afb9-4ae3-42d8-bcc4-1827361b23ed None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:26:11.855288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:11.855288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:11.855288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:11.855288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:11.855288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:11.855288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:11.855288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:11.855288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:11.855288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:11.855288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:11.855288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:11.855288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:11.855288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:11.855288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:11.855288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:11.856498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:11.856498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:11.856498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:11.856498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:11.856498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:11.856498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:11.856498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:11.856498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:11.856498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:11.856498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:11.856498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:11.856498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:11.856498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:11.856498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:11.856498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:11.856498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:11.856498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:11.856498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:11.857603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:11.857603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:11.857603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:11.857603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:11.857603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:11.857603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:11.857603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:11.857603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:11.857603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:26:11.857603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:11.859572 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 330/684] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:26:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:11.860466 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572afb9-4ae3-42d8-bcc4-1827361b23ed None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:11.870808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30988f63-843e-4f18-9a03-880885c542b0 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:11.918129 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df89823-df3a-4e8c-97f2-063f08754d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:11.919105 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df89823-df3a-4e8c-97f2-063f08754d68 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:11.920071 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df89823-df3a-4e8c-97f2-063f08754d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:11.920524 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df89823-df3a-4e8c-97f2-063f08754d68 None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:11.921766 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df89823-df3a-4e8c-97f2-063f08754d68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:11.922236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df89823-df3a-4e8c-97f2-063f08754d68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:11.924841 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df89823-df3a-4e8c-97f2-063f08754d68 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:11.945966 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 355/685] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:11.945966 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df89823-df3a-4e8c-97f2-063f08754d68 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:11.976971 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572afb9-4ae3-42d8-bcc4-1827361b23ed None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:12.013889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7069b7c-7cb9-4fd5-9482-b70bdbef7674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:12.014745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7069b7c-7cb9-4fd5-9482-b70bdbef7674 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:12.014973 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7069b7c-7cb9-4fd5-9482-b70bdbef7674 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:12.015248 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7069b7c-7cb9-4fd5-9482-b70bdbef7674 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:12.015990 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7069b7c-7cb9-4fd5-9482-b70bdbef7674 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:12.016300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7069b7c-7cb9-4fd5-9482-b70bdbef7674 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:12.017657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7069b7c-7cb9-4fd5-9482-b70bdbef7674 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:12.026998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 331/686] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:26:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:12.027880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7069b7c-7cb9-4fd5-9482-b70bdbef7674 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:12.081667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df89823-df3a-4e8c-97f2-063f08754d68 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:12.086084 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:12.092846 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:12.131554 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78207f-d717-46b9-95d9-233de75c0fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:12.132222 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78207f-d717-46b9-95d9-233de75c0fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:12.132501 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78207f-d717-46b9-95d9-233de75c0fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:12.132771 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78207f-d717-46b9-95d9-233de75c0fa4 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:12.133605 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78207f-d717-46b9-95d9-233de75c0fa4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:12.133911 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78207f-d717-46b9-95d9-233de75c0fa4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:12.135630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78207f-d717-46b9-95d9-233de75c0fa4 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:12.154209 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 356/687] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Jun 23 11:26:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:12.155145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78207f-d717-46b9-95d9-233de75c0fa4 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:12.156229 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7069b7c-7cb9-4fd5-9482-b70bdbef7674 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:12.197754 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c59e2c-09ec-47e6-8e3c-6f8347f9cdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:12.198443 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c59e2c-09ec-47e6-8e3c-6f8347f9cdef None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:12.198775 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c59e2c-09ec-47e6-8e3c-6f8347f9cdef None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:12.199063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c59e2c-09ec-47e6-8e3c-6f8347f9cdef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:12.200581 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c59e2c-09ec-47e6-8e3c-6f8347f9cdef None admin] RBAC: Adding query filter params (role.id=13708a4819a94b94b47ba5994aaa9310, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=810fa3f54cd04f989252765f7e76e602) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:12.200899 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c59e2c-09ec-47e6-8e3c-6f8347f9cdef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:12.202418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c59e2c-09ec-47e6-8e3c-6f8347f9cdef None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:12.214562 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 332/688] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:26:12 2022] GET /identity/v3/role_assignments?user.id=810fa3f54cd04f989252765f7e76e602&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=13708a4819a94b94b47ba5994aaa9310 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:12.214562 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c59e2c-09ec-47e6-8e3c-6f8347f9cdef None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:12.288051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78207f-d717-46b9-95d9-233de75c0fa4 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:12.331144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59b38f-0b42-44b2-b169-caed60ef7cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:12.332580 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59b38f-0b42-44b2-b169-caed60ef7cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:12.333413 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59b38f-0b42-44b2-b169-caed60ef7cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:12.333903 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59b38f-0b42-44b2-b169-caed60ef7cc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:12.346728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59b38f-0b42-44b2-b169-caed60ef7cc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:12.349268 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59b38f-0b42-44b2-b169-caed60ef7cc1 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:12.372614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-7d59b38f-0b42-44b2-b169-caed60ef7cc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:12.372614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:12.372614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:12.372614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:12.372614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:12.372614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:12.372614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:12.372614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:12.372614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:12.372614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:12.372614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:12.372614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:12.372614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:12.372614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:12.372614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:12.373590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:12.373590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:12.373590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:12.373590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:12.373590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:12.373590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:12.373590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:12.373590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:12.373590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:12.373590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:12.373590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:12.373590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:12.373590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:12.373590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:12.373590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:12.373590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:12.373590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:12.374496 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:12.374496 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:12.374496 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:12.374496 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:12.377979 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 357/689] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:12.383122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59b38f-0b42-44b2-b169-caed60ef7cc1 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:12.422973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:12.437714 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c59e2c-09ec-47e6-8e3c-6f8347f9cdef None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:12.449788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:12.483446 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be50457-69ba-451f-9940-8940c72c62c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:12.484332 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be50457-69ba-451f-9940-8940c72c62c1 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:12.484584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be50457-69ba-451f-9940-8940c72c62c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:12.484913 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be50457-69ba-451f-9940-8940c72c62c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:12.550892 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59b38f-0b42-44b2-b169-caed60ef7cc1 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:12.580572 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be50457-69ba-451f-9940-8940c72c62c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:12.582085 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be50457-69ba-451f-9940-8940c72c62c1 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:12.608014 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85060c6c-bfd2-4aa8-a660-641f80115bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:12.611619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85060c6c-bfd2-4aa8-a660-641f80115bbf None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:12.611911 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85060c6c-bfd2-4aa8-a660-641f80115bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:12.612197 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85060c6c-bfd2-4aa8-a660-641f80115bbf None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:12.612951 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85060c6c-bfd2-4aa8-a660-641f80115bbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:12.613264 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85060c6c-bfd2-4aa8-a660-641f80115bbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:12.614830 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85060c6c-bfd2-4aa8-a660-641f80115bbf None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:12.630118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 358/690] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:12.635150 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85060c6c-bfd2-4aa8-a660-641f80115bbf None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:12.728322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 333/691] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:26:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:26:12.736741 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be50457-69ba-451f-9940-8940c72c62c1 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:12.753111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:12.774905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:12.816257 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85060c6c-bfd2-4aa8-a660-641f80115bbf None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:12.871024 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be50457-69ba-451f-9940-8940c72c62c1 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:12.906802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de624c-23f9-4a31-b6bb-ab729230c1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:12.907776 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de624c-23f9-4a31-b6bb-ab729230c1aa None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:12.908070 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de624c-23f9-4a31-b6bb-ab729230c1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:12.908347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de624c-23f9-4a31-b6bb-ab729230c1aa None admin] PATH_INFO: `/v3/users/nova` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:12.917272 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de624c-23f9-4a31-b6bb-ab729230c1aa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:12.919214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de624c-23f9-4a31-b6bb-ab729230c1aa None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:12.934871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-06de624c-23f9-4a31-b6bb-ab729230c1aa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:26:12.934871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:12.934871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:12.934871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:12.934871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:12.934871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:12.934871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:12.934871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:12.934871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:12.934871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:12.934871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:12.934871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:12.934871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:12.934871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:12.934871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:12.934871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:12.935847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:12.935847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:12.935847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:12.935847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:12.935847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:12.935847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:12.935847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:12.935847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:12.935847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:12.935847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:12.935847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:12.935847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:12.935847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:12.935847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:12.935847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:12.935847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:12.935847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:12.935847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:12.937236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:12.937236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:12.937236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:12.937236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:12.937236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:12.937236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:12.937236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:12.937236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:12.937236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:26:12.937236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:12.937236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 359/692] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jun 23 11:26:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:12.941647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de624c-23f9-4a31-b6bb-ab729230c1aa None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:12.942059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009328b-5510-413c-8a84-92118215d0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:12.943312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009328b-5510-413c-8a84-92118215d0fc None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:12.943764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009328b-5510-413c-8a84-92118215d0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:12.944180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009328b-5510-413c-8a84-92118215d0fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:12.968636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009328b-5510-413c-8a84-92118215d0fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:12.970323 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009328b-5510-413c-8a84-92118215d0fc None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:13.011357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-3009328b-5510-413c-8a84-92118215d0fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:13.011357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:13.011357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:13.011357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:13.011357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:13.011357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:13.011357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:13.011357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:13.011357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:13.011357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:13.011357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:13.011357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:13.011357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:13.011357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:13.011357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:13.013283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:13.013283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:13.013283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:13.013283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:13.013283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:13.013283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:13.013283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:13.013283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:13.013283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:13.013283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:13.013283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:13.013283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:13.013283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:13.013283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:13.013283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:13.013283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:13.013283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:13.014166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:13.014166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:13.014166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:13.014166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:13.014166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 334/693] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:13.017418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009328b-5510-413c-8a84-92118215d0fc None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:13.087241 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:13.134278 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009328b-5510-413c-8a84-92118215d0fc None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:13.156738 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de624c-23f9-4a31-b6bb-ab729230c1aa None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:13.174778 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01de82-6baa-4994-a743-6a03bab98b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:13.175455 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01de82-6baa-4994-a743-6a03bab98b93 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:13.176172 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01de82-6baa-4994-a743-6a03bab98b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:13.176463 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01de82-6baa-4994-a743-6a03bab98b93 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:13.177358 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b01de82-6baa-4994-a743-6a03bab98b93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:13.177678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b01de82-6baa-4994-a743-6a03bab98b93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:13.179091 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b01de82-6baa-4994-a743-6a03bab98b93 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:13.193652 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 335/694] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:13.199488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01de82-6baa-4994-a743-6a03bab98b93 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:13.230619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b84cc8-1957-495a-9fdc-7bb137eb283a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:13.231886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b84cc8-1957-495a-9fdc-7bb137eb283a None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:13.232158 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b84cc8-1957-495a-9fdc-7bb137eb283a None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:13.232415 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b84cc8-1957-495a-9fdc-7bb137eb283a None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:13.233256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b84cc8-1957-495a-9fdc-7bb137eb283a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:13.233562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b84cc8-1957-495a-9fdc-7bb137eb283a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:13.236354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b84cc8-1957-495a-9fdc-7bb137eb283a None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:13.242716 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:13.265259 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 360/695] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jun 23 11:26:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:13.271256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b84cc8-1957-495a-9fdc-7bb137eb283a None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:13.410670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01de82-6baa-4994-a743-6a03bab98b93 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:13.447856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:13.453685 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b84cc8-1957-495a-9fdc-7bb137eb283a None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:13.476449 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ade29-a2b1-4606-a072-bfe9a61eb6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:13.478734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ade29-a2b1-4606-a072-bfe9a61eb6fe None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:13.479256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ade29-a2b1-4606-a072-bfe9a61eb6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:13.480598 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ade29-a2b1-4606-a072-bfe9a61eb6fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:13.498414 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ade29-a2b1-4606-a072-bfe9a61eb6fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:13.500240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ade29-a2b1-4606-a072-bfe9a61eb6fe None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:13.523762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-a69ade29-a2b1-4606-a072-bfe9a61eb6fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:13.523762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:13.523762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:13.523762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:13.523762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:13.523762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:13.523762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:13.523762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:13.523762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:13.523762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:13.523762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:13.523762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:13.523762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:13.523762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:13.523762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:13.524709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:13.524709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:13.524709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:13.524709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:13.524709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:13.524709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:13.524709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:13.524709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:13.524709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:13.524709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:13.524709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:13.524709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:13.524709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:13.524709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:13.524709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:13.524709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:13.524709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:13.525621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:13.525621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:13.525621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:13.525621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:13.525621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:13.525621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:13.525621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:13.525621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:13.531765 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae53845-db8b-4eab-b664-d296b6fbcb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:13.532689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae53845-db8b-4eab-b664-d296b6fbcb70 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:13.533182 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae53845-db8b-4eab-b664-d296b6fbcb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:13.533666 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae53845-db8b-4eab-b664-d296b6fbcb70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:13.538744 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 336/696] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:13.544661 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ade29-a2b1-4606-a072-bfe9a61eb6fe None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:13.645284 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:13.653787 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae53845-db8b-4eab-b664-d296b6fbcb70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:13.658675 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae53845-db8b-4eab-b664-d296b6fbcb70 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:13.668565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ade29-a2b1-4606-a072-bfe9a61eb6fe None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:13.706187 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20963923-82da-4423-a0ff-9b71a18e36eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:13.706875 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20963923-82da-4423-a0ff-9b71a18e36eb None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:13.707158 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20963923-82da-4423-a0ff-9b71a18e36eb None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:13.707431 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20963923-82da-4423-a0ff-9b71a18e36eb None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:13.708211 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20963923-82da-4423-a0ff-9b71a18e36eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:13.708523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20963923-82da-4423-a0ff-9b71a18e36eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:13.709971 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20963923-82da-4423-a0ff-9b71a18e36eb None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:13.722177 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 337/697] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:13.723465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c552a4-5038-4809-bcd1-c4f00a014b81 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:13.723753 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c552a4-5038-4809-bcd1-c4f00a014b81 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:13.724032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c552a4-5038-4809-bcd1-c4f00a014b81 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:13.725407 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 338/698] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:13.730190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977d7cb-21a9-4e83-97e1-3ff490994b02 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:13.730482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977d7cb-21a9-4e83-97e1-3ff490994b02 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:13.730787 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977d7cb-21a9-4e83-97e1-3ff490994b02 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:13.734660 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 339/699] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:13.734660 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977d7cb-21a9-4e83-97e1-3ff490994b02 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:13.773196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 361/700] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:26:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:26:13.774984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdde29d-0012-43c5-93ac-4e124582adc9 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:13.775417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdde29d-0012-43c5-93ac-4e124582adc9 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:13.775804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdde29d-0012-43c5-93ac-4e124582adc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:13.788034 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:13.795078 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-acdde29d-0012-43c5-93ac-4e124582adc9 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:13.796365 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-acdde29d-0012-43c5-93ac-4e124582adc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:13.851209 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977d7cb-21a9-4e83-97e1-3ff490994b02 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:13.870456 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 362/701] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:26:13.871934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211607c-d6ab-4d41-983c-e6c8ad25e0f9 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:13.872188 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211607c-d6ab-4d41-983c-e6c8ad25e0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:13.872596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211607c-d6ab-4d41-983c-e6c8ad25e0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:13.888903 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-5211607c-d6ab-4d41-983c-e6c8ad25e0f9 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:13.890079 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-5211607c-d6ab-4d41-983c-e6c8ad25e0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:13.892935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da539518-6ba5-43c9-bf45-7ee305dc6db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:13.893897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da539518-6ba5-43c9-bf45-7ee305dc6db0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:13.894396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da539518-6ba5-43c9-bf45-7ee305dc6db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:13.894722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da539518-6ba5-43c9-bf45-7ee305dc6db0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:13.895558 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da539518-6ba5-43c9-bf45-7ee305dc6db0 None admin] RBAC: Adding query filter params (role.id=13708a4819a94b94b47ba5994aaa9310, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=450b1fad96604fb09d0705895c6d3da4) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:13.895874 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da539518-6ba5-43c9-bf45-7ee305dc6db0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:13.897298 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da539518-6ba5-43c9-bf45-7ee305dc6db0 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:13.904339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 340/702] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:26:13 2022] GET /identity/v3/role_assignments?user.id=450b1fad96604fb09d0705895c6d3da4&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=13708a4819a94b94b47ba5994aaa9310 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:13.905362 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da539518-6ba5-43c9-bf45-7ee305dc6db0 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:13.908939 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:13.983075 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 363/703] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:26:13.984230 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211607c-d6ab-4d41-983c-e6c8ad25e0f9 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:14.050102 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da539518-6ba5-43c9-bf45-7ee305dc6db0 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:14.053379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:14.096631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce7a50-381a-4ea1-9e2a-958ed01f24e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:14.097459 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce7a50-381a-4ea1-9e2a-958ed01f24e0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:14.097750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce7a50-381a-4ea1-9e2a-958ed01f24e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:14.098708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce7a50-381a-4ea1-9e2a-958ed01f24e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:14.109413 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce7a50-381a-4ea1-9e2a-958ed01f24e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:14.111334 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce7a50-381a-4ea1-9e2a-958ed01f24e0 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:14.115784 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211607c-d6ab-4d41-983c-e6c8ad25e0f9 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:14.130619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-96ce7a50-381a-4ea1-9e2a-958ed01f24e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:14.130619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:14.130619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:14.130619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:14.130619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:14.130619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:14.130619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:14.130619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:14.130619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:14.130619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:14.130619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:14.130619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:14.130619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:14.130619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:14.130619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:14.131540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:14.131540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:14.131540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:14.131540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:14.131540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:14.131540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:14.131540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:14.131540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:14.131540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:14.131540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:14.131540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:14.131540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:14.131540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:14.131540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:14.131540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:14.131540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:14.131540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:14.132481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:14.132481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:14.132481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:14.132481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:14.132481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 341/704] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:14.132481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce7a50-381a-4ea1-9e2a-958ed01f24e0 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:14.161295 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0212c13-3632-4ca4-b255-c59772e1b916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:14.162309 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0212c13-3632-4ca4-b255-c59772e1b916 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:14.162683 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0212c13-3632-4ca4-b255-c59772e1b916 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:14.162962 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0212c13-3632-4ca4-b255-c59772e1b916 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:14.174101 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0212c13-3632-4ca4-b255-c59772e1b916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:14.175595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0212c13-3632-4ca4-b255-c59772e1b916 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:14.196088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-a0212c13-3632-4ca4-b255-c59772e1b916 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:14.196088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:14.196088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:14.196088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:14.196088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:14.196088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:14.196088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:14.196088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:14.196088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:14.196088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:14.196088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:14.196088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:14.196088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:14.196088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:14.196088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:14.197011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:14.197011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:14.197011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:14.197011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:14.197011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:14.197011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:14.197011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:14.197011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:14.197011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:14.197011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:14.197011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:14.197011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:14.197011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:14.197011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:14.197011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:14.197011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:14.197011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:14.197927 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:14.197927 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:14.197927 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:14.197927 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:14.197927 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 364/705] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:14.198371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0212c13-3632-4ca4-b255-c59772e1b916 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:14.208439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:14.250436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce7a50-381a-4ea1-9e2a-958ed01f24e0 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:14.305707 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb1c96-2a95-41ec-813e-ede940986847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:14.306558 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb1c96-2a95-41ec-813e-ede940986847 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:14.306849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb1c96-2a95-41ec-813e-ede940986847 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:14.307125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb1c96-2a95-41ec-813e-ede940986847 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:14.307781 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb1c96-2a95-41ec-813e-ede940986847 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:14.309158 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb1c96-2a95-41ec-813e-ede940986847 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:14.324610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-14bb1c96-2a95-41ec-813e-ede940986847 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 11:26:14.324610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:14.324610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:14.324610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:14.324610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:14.324610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:14.324610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:14.324610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:14.324610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 11:26:14.324610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:14.324610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:14.324610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:14.324610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:14.324610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:14.324610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:14.324610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:14.326499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:14.326499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:14.326499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:14.326499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:14.326499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:14.326499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:14.326499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:14.326499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:14.326499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:14.326499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:14.326499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:14.326499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:14.326499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:14.326499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:14.326499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:14.326499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:14.327364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:14.327364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:14.327364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:14.327364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:14.327364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:14.327364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:14.327364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:14.327364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:14.327364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:14.327364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:14.327364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:14.327364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:14.327364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:14.327364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 11:26:14.327364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:14.327364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 342/706] 10.222.0.13 () {56 vars in 1129 bytes} [Thu Jun 23 11:26:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:14.328218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0212c13-3632-4ca4-b255-c59772e1b916 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:14.328291 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb1c96-2a95-41ec-813e-ede940986847 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:14.355264 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:14.380927 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ab0f7-fcfd-4b54-9e1b-ab6375152c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:14.382140 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ab0f7-fcfd-4b54-9e1b-ab6375152c82 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:14.382708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ab0f7-fcfd-4b54-9e1b-ab6375152c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:14.383537 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ab0f7-fcfd-4b54-9e1b-ab6375152c82 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:14.384712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ab0f7-fcfd-4b54-9e1b-ab6375152c82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:14.385248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ab0f7-fcfd-4b54-9e1b-ab6375152c82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:14.387297 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ab0f7-fcfd-4b54-9e1b-ab6375152c82 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:14.405395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 365/707] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:14.406416 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ab0f7-fcfd-4b54-9e1b-ab6375152c82 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:14.452196 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb1c96-2a95-41ec-813e-ede940986847 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:14.479090 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:14.495982 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac9892-c7b0-4eb0-b3e1-fc7531c5fbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:14.497065 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac9892-c7b0-4eb0-b3e1-fc7531c5fbfd None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:14.497548 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac9892-c7b0-4eb0-b3e1-fc7531c5fbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:14.498018 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac9892-c7b0-4eb0-b3e1-fc7531c5fbfd None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:14.498978 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac9892-c7b0-4eb0-b3e1-fc7531c5fbfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:14.499506 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac9892-c7b0-4eb0-b3e1-fc7531c5fbfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:14.501318 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac9892-c7b0-4eb0-b3e1-fc7531c5fbfd None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:14.514471 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 343/708] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:14.515646 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac9892-c7b0-4eb0-b3e1-fc7531c5fbfd None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:14.526470 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ab0f7-fcfd-4b54-9e1b-ab6375152c82 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:14.565678 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c431e8-8e16-41b5-b5b9-442e06997ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:14.566395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c431e8-8e16-41b5-b5b9-442e06997ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:14.566769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c431e8-8e16-41b5-b5b9-442e06997ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:14.567055 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c431e8-8e16-41b5-b5b9-442e06997ad4 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:14.568155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c431e8-8e16-41b5-b5b9-442e06997ad4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:14.568426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c431e8-8e16-41b5-b5b9-442e06997ad4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:14.569949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c431e8-8e16-41b5-b5b9-442e06997ad4 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:14.580473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 366/709] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Jun 23 11:26:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:14.581419 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c431e8-8e16-41b5-b5b9-442e06997ad4 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:14.605767 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:14.627731 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac9892-c7b0-4eb0-b3e1-fc7531c5fbfd None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:14.664322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ff8f0-abea-448b-b435-7c52c82363c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:14.665512 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ff8f0-abea-448b-b435-7c52c82363c4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:14.665993 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ff8f0-abea-448b-b435-7c52c82363c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:14.666463 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ff8f0-abea-448b-b435-7c52c82363c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:14.674188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ff8f0-abea-448b-b435-7c52c82363c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:14.675911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ff8f0-abea-448b-b435-7c52c82363c4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:14.690373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-ed7ff8f0-abea-448b-b435-7c52c82363c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:14.690373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:14.690373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:14.690373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:14.690373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:14.690373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:14.690373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:14.690373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:14.690373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:14.690373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:14.690373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:14.690373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:14.690373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:14.690373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:14.690373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:14.691313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:14.691313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:14.691313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:14.691313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:14.691313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:14.691313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:14.691313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:14.691313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:14.691313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:14.691313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:14.691313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:14.691313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:14.691313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:14.691313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:14.691313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:14.691313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:14.691313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:14.692227 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:14.692227 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:14.692227 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:14.692227 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:14.692227 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:14.692227 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:14.692227 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:14.692227 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:14.699059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c431e8-8e16-41b5-b5b9-442e06997ad4 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:14.699907 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 344/710] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:14.700985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ff8f0-abea-448b-b435-7c52c82363c4 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:14.729774 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:14.740102 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e47c44-4530-4749-8a6a-0e89146d11c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:14.740772 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e47c44-4530-4749-8a6a-0e89146d11c6 None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:14.741883 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e47c44-4530-4749-8a6a-0e89146d11c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:14.741883 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e47c44-4530-4749-8a6a-0e89146d11c6 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:14.742971 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e47c44-4530-4749-8a6a-0e89146d11c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:14.744677 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e47c44-4530-4749-8a6a-0e89146d11c6 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:14.804816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 367/711] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 23 11:26:14 2022] POST /identity/v3/users => generated 254 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:26:14.805961 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e47c44-4530-4749-8a6a-0e89146d11c6 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:14.821362 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ff8f0-abea-448b-b435-7c52c82363c4 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:14.867122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf16b8-aaf0-4684-a3f1-23d19e54b7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:14.868213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf16b8-aaf0-4684-a3f1-23d19e54b7fd None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:14.868682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf16b8-aaf0-4684-a3f1-23d19e54b7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:14.869290 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf16b8-aaf0-4684-a3f1-23d19e54b7fd None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:14.870255 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcf16b8-aaf0-4684-a3f1-23d19e54b7fd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:14.870774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcf16b8-aaf0-4684-a3f1-23d19e54b7fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:14.874198 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcf16b8-aaf0-4684-a3f1-23d19e54b7fd None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:14.889166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 345/712] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Jun 23 11:26:14 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:14.891138 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae8f72-0aca-42f6-a4ee-fbe865762f75 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:14.891675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae8f72-0aca-42f6-a4ee-fbe865762f75 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:14.892181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae8f72-0aca-42f6-a4ee-fbe865762f75 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:14.894365 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 346/713] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:14.900285 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b1dae-3929-42f0-9229-29e86cb43d13 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:14.900928 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b1dae-3929-42f0-9229-29e86cb43d13 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:14.901384 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b1dae-3929-42f0-9229-29e86cb43d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:14.922935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:14.937064 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a09b1dae-3929-42f0-9229-29e86cb43d13 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:14.938412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a09b1dae-3929-42f0-9229-29e86cb43d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:15.046583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 347/714] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:26:15.060979 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e47c44-4530-4749-8a6a-0e89146d11c6 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:15.076980 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b1dae-3929-42f0-9229-29e86cb43d13 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:15.109286 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7b9cd-d87d-4149-ab33-3a27b8af00f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:15.110242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7b9cd-d87d-4149-ab33-3a27b8af00f1 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:15.110544 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7b9cd-d87d-4149-ab33-3a27b8af00f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:15.110837 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7b9cd-d87d-4149-ab33-3a27b8af00f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:15.111829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7b9cd-d87d-4149-ab33-3a27b8af00f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:15.113429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7b9cd-d87d-4149-ab33-3a27b8af00f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:15.115205 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7b9cd-d87d-4149-ab33-3a27b8af00f1 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:15.130977 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 368/715] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:15.136636 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7b9cd-d87d-4149-ab33-3a27b8af00f1 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:15.202806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:15.212761 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b1dae-3929-42f0-9229-29e86cb43d13 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:15.255929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec18541-9c10-4b61-9e78-28c3aa65812d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:15.256726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec18541-9c10-4b61-9e78-28c3aa65812d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:15.257007 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec18541-9c10-4b61-9e78-28c3aa65812d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:15.257314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec18541-9c10-4b61-9e78-28c3aa65812d None admin] PATH_INFO: `/v3/services/network` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:15.257974 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec18541-9c10-4b61-9e78-28c3aa65812d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:15.262051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec18541-9c10-4b61-9e78-28c3aa65812d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:15.265717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:15.271789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7b9cd-d87d-4149-ab33-3a27b8af00f1 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:15.277011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-4ec18541-9c10-4b61-9e78-28c3aa65812d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 11:26:15.277011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:15.277011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:15.277011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:15.277011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:15.277011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:15.277011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:15.277011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:15.277011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 11:26:15.277011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:15.277011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:15.277011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:15.277011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:15.277011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:15.277011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:15.277011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:15.277011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:15.278206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:15.278206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:15.278206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:15.278206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:15.278206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:15.278206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:15.278206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:15.278206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:15.278206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:15.278206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:15.278206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:15.278206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:15.278206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:15.278206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:15.278206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:15.278206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:15.279270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:15.279270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:15.279270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:15.279270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:15.279270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:15.279270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:15.279270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:15.279270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:15.279270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:15.279270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:15.279270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:15.279270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:15.279270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 11:26:15.279270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:15.281646 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 348/716] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:15 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:15.288071 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec18541-9c10-4b61-9e78-28c3aa65812d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:15.340450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3109d-39f2-406f-aa92-81a981b05130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:15.341470 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3109d-39f2-406f-aa92-81a981b05130 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:15.341930 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3109d-39f2-406f-aa92-81a981b05130 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:15.342329 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3109d-39f2-406f-aa92-81a981b05130 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:15.343319 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3109d-39f2-406f-aa92-81a981b05130 None admin] RBAC: Adding query filter params (role.id=c9e006de0d6c4812ab29d778e9c8a830, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=161c5ed1e47e450290a1f428ee100b68) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:15.346991 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3109d-39f2-406f-aa92-81a981b05130 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:15.348608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3109d-39f2-406f-aa92-81a981b05130 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:15.358634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 369/717] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:26:15 2022] GET /identity/v3/role_assignments?user.id=161c5ed1e47e450290a1f428ee100b68&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=c9e006de0d6c4812ab29d778e9c8a830 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:15.461018 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec18541-9c10-4b61-9e78-28c3aa65812d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:15.510486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732adbf8-abe9-424e-9bb6-365c345f26a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:15.511788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732adbf8-abe9-424e-9bb6-365c345f26a9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:15.512287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732adbf8-abe9-424e-9bb6-365c345f26a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:15.513286 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732adbf8-abe9-424e-9bb6-365c345f26a9 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:15.514245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732adbf8-abe9-424e-9bb6-365c345f26a9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:15.514944 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732adbf8-abe9-424e-9bb6-365c345f26a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:15.516590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732adbf8-abe9-424e-9bb6-365c345f26a9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:15.530565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 349/718] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 23 11:26:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:15.535791 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3109d-39f2-406f-aa92-81a981b05130 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:15.648642 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3109d-39f2-406f-aa92-81a981b05130 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:15.660630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:15.684304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60157f-6e8a-4364-b4e8-4b25e2bd8df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:15.685490 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60157f-6e8a-4364-b4e8-4b25e2bd8df1 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:15.685771 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60157f-6e8a-4364-b4e8-4b25e2bd8df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:15.686046 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60157f-6e8a-4364-b4e8-4b25e2bd8df1 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:15.686777 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60157f-6e8a-4364-b4e8-4b25e2bd8df1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:15.687083 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60157f-6e8a-4364-b4e8-4b25e2bd8df1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:15.688447 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60157f-6e8a-4364-b4e8-4b25e2bd8df1 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:15.694902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 370/719] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 23 11:26:15 2022] GET /identity/v3/services?type=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:16.262815 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2069d8d-bedb-4dd9-a431-e219f3c992c0 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:16.263435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2069d8d-bedb-4dd9-a431-e219f3c992c0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:16.263926 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2069d8d-bedb-4dd9-a431-e219f3c992c0 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:16.265706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 350/720] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:16.270697 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503a386-5001-4357-a9ed-dfe48f09d67d None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:16.271482 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503a386-5001-4357-a9ed-dfe48f09d67d None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:16.271907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503a386-5001-4357-a9ed-dfe48f09d67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:16.299543 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c503a386-5001-4357-a9ed-dfe48f09d67d None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:16.301024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c503a386-5001-4357-a9ed-dfe48f09d67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:16.391721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 371/721] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:26:16.427213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2069d8d-bedb-4dd9-a431-e219f3c992c0 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:16.510752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:16.564399 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2069d8d-bedb-4dd9-a431-e219f3c992c0 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:16.609115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d07d92-4814-4be1-a008-8cf551d2cee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:16.609922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d07d92-4814-4be1-a008-8cf551d2cee4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:16.610211 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d07d92-4814-4be1-a008-8cf551d2cee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:16.610492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d07d92-4814-4be1-a008-8cf551d2cee4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:16.625859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d07d92-4814-4be1-a008-8cf551d2cee4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:16.637128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-f8d07d92-4814-4be1-a008-8cf551d2cee4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:26:16.637128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:16.637128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:16.637128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:16.637128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:16.637128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:16.637128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:16.637128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:16.637128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:16.637128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:16.637128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:16.637128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:16.637128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:16.637128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:16.637128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:16.637128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:16.638657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:16.638657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:16.638657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:16.638657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:16.638657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:16.638657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:16.638657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:16.638657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:16.638657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:16.638657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:16.638657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:16.638657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:16.638657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:16.638657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:16.638657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:16.638657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:16.639538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:16.639538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:16.639538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:16.639538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:16.639538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:16.639538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:16.639538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:16.639538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:16.639538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:16.639538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:16.639538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:26:16.639538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:16.639538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 351/722] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:26:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:16.647511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503a386-5001-4357-a9ed-dfe48f09d67d None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:16.801872 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503a386-5001-4357-a9ed-dfe48f09d67d None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:16.805632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:16.846570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e68277-58b9-44dd-b6c1-33becd72ba94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:16.847258 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e68277-58b9-44dd-b6c1-33becd72ba94 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:16.847545 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e68277-58b9-44dd-b6c1-33becd72ba94 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:16.847821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e68277-58b9-44dd-b6c1-33becd72ba94 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:16.848567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e68277-58b9-44dd-b6c1-33becd72ba94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:16.848875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e68277-58b9-44dd-b6c1-33becd72ba94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:16.850245 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e68277-58b9-44dd-b6c1-33becd72ba94 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:16.864530 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 372/723] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:26:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:16.870003 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d07d92-4814-4be1-a008-8cf551d2cee4 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:16.993341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d07d92-4814-4be1-a008-8cf551d2cee4 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:17.017338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:17.034254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62017493-47b7-4286-978c-c16a0b948f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:17.035082 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62017493-47b7-4286-978c-c16a0b948f42 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:17.035375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62017493-47b7-4286-978c-c16a0b948f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:17.035651 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62017493-47b7-4286-978c-c16a0b948f42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:17.055092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62017493-47b7-4286-978c-c16a0b948f42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:17.056552 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62017493-47b7-4286-978c-c16a0b948f42 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:17.078691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-62017493-47b7-4286-978c-c16a0b948f42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:17.078691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:17.078691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:17.078691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:17.078691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:17.078691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:17.078691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:17.078691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:17.078691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:17.078691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:17.078691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:17.078691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:17.078691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:17.078691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:17.078691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:17.079615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:17.079615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:17.079615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:17.079615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:17.079615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:17.079615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:17.079615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:17.079615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:17.079615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:17.079615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:17.079615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:17.079615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:17.079615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:17.079615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:17.079615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:17.079615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:17.079615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:17.080488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:17.080488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:17.080488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:17.080488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:17.080488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 352/724] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:17.086784 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e68277-58b9-44dd-b6c1-33becd72ba94 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:17.229344 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e68277-58b9-44dd-b6c1-33becd72ba94 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:17.284676 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc84c0-d8b0-4100-899a-b62fc255ca5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:17.285460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc84c0-d8b0-4100-899a-b62fc255ca5a None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:17.285641 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc84c0-d8b0-4100-899a-b62fc255ca5a None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:17.285921 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc84c0-d8b0-4100-899a-b62fc255ca5a None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:17.286684 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc84c0-d8b0-4100-899a-b62fc255ca5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:17.286996 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc84c0-d8b0-4100-899a-b62fc255ca5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:17.288362 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc84c0-d8b0-4100-899a-b62fc255ca5a None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:17.306580 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 373/725] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:17.315718 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62017493-47b7-4286-978c-c16a0b948f42 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:17.359862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30a8ca-3f62-48a1-8ac5-aa64d49a4579 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:17.360506 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30a8ca-3f62-48a1-8ac5-aa64d49a4579 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:17.361009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30a8ca-3f62-48a1-8ac5-aa64d49a4579 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:17.362870 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 374/726] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:17.368914 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2b791-b4a6-4f12-8a16-af8e426ffb49 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:17.369436 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2b791-b4a6-4f12-8a16-af8e426ffb49 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:17.369914 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2b791-b4a6-4f12-8a16-af8e426ffb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:17.395053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-f6d2b791-b4a6-4f12-8a16-af8e426ffb49 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:17.397906 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f6d2b791-b4a6-4f12-8a16-af8e426ffb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:17.440573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:17.509730 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62017493-47b7-4286-978c-c16a0b948f42 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:17.511237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 375/727] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:26:17.560178 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2b791-b4a6-4f12-8a16-af8e426ffb49 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:17.572128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85d2af-9fa8-4d41-84f8-ddc927a8e29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:17.573467 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85d2af-9fa8-4d41-84f8-ddc927a8e29b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:17.575478 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85d2af-9fa8-4d41-84f8-ddc927a8e29b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:17.577240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85d2af-9fa8-4d41-84f8-ddc927a8e29b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:17.595357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85d2af-9fa8-4d41-84f8-ddc927a8e29b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:17.597028 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85d2af-9fa8-4d41-84f8-ddc927a8e29b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:17.621072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-fc85d2af-9fa8-4d41-84f8-ddc927a8e29b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:26:17.621072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:17.621072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:17.621072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:17.621072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:17.621072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:17.621072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:17.621072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:17.621072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:17.621072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:17.621072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:17.621072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:17.621072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:17.621072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:17.621072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:17.621072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:17.623010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:17.623010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:17.623010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:17.623010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:17.623010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:17.623010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:17.623010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:17.623010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:17.623010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:17.623010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:17.623010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:17.623010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:17.623010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:17.623010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:17.623010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:17.623010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:17.623010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:17.623010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:17.624182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:17.624182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:17.624182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:17.624182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:17.624182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:17.624182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:17.624182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:17.624182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:17.624182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:26:17.624182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:17.626939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 353/728] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:26:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:17.650505 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85d2af-9fa8-4d41-84f8-ddc927a8e29b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:17.731490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:17.778459 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2b791-b4a6-4f12-8a16-af8e426ffb49 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:17.826957 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728f4f1-8bfc-4436-9d07-bee2a84626ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:17.828076 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728f4f1-8bfc-4436-9d07-bee2a84626ab None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:17.828554 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728f4f1-8bfc-4436-9d07-bee2a84626ab None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:17.828867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728f4f1-8bfc-4436-9d07-bee2a84626ab None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:17.829622 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728f4f1-8bfc-4436-9d07-bee2a84626ab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:17.831677 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728f4f1-8bfc-4436-9d07-bee2a84626ab None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:17.867012 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85d2af-9fa8-4d41-84f8-ddc927a8e29b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:17.909954 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b792407-3107-4a5d-9da2-c51ca28cce47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:17.910676 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b792407-3107-4a5d-9da2-c51ca28cce47 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:17.910963 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b792407-3107-4a5d-9da2-c51ca28cce47 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:17.911241 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b792407-3107-4a5d-9da2-c51ca28cce47 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:17.912092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b792407-3107-4a5d-9da2-c51ca28cce47 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:17.912407 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b792407-3107-4a5d-9da2-c51ca28cce47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:17.913856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b792407-3107-4a5d-9da2-c51ca28cce47 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:17.939174 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 354/729] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Jun 23 11:26:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:17.940894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56cbfe-b677-4645-9769-d78a5e612255 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:17.941177 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56cbfe-b677-4645-9769-d78a5e612255 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:17.941484 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56cbfe-b677-4645-9769-d78a5e612255 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:17.943400 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 355/730] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:17.950743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56cbfe-b677-4645-9769-d78a5e612255 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:17.952504 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 376/731] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 23 11:26:17 2022] POST /identity/v3/services => generated 251 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:26:17.955417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd90215-1f08-4459-bef0-8ee95e43b495 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:17.956766 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd90215-1f08-4459-bef0-8ee95e43b495 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:17.961101 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd90215-1f08-4459-bef0-8ee95e43b495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:17.988025 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-edd90215-1f08-4459-bef0-8ee95e43b495 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:17.989264 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-edd90215-1f08-4459-bef0-8ee95e43b495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:18.006086 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:18.111222 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 377/732] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:26:18.115074 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc485b3-ea21-4305-9b2c-2bd0696c39d3 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:18.115369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc485b3-ea21-4305-9b2c-2bd0696c39d3 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:18.115652 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc485b3-ea21-4305-9b2c-2bd0696c39d3 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:18.117389 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 378/733] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:18.126754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd108e64-3103-4cb5-ac6a-e9ca4a32b42d None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:18.127091 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd108e64-3103-4cb5-ac6a-e9ca4a32b42d None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:18.127316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd108e64-3103-4cb5-ac6a-e9ca4a32b42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:18.154388 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-dd108e64-3103-4cb5-ac6a-e9ca4a32b42d None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:18.156555 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-dd108e64-3103-4cb5-ac6a-e9ca4a32b42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:18.173163 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:18.183856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56cbfe-b677-4645-9769-d78a5e612255 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:18.247501 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc952bc3-ed3f-4852-bb2c-905e60109595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:18.248182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc952bc3-ed3f-4852-bb2c-905e60109595 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:18.248469 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc952bc3-ed3f-4852-bb2c-905e60109595 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:18.248750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc952bc3-ed3f-4852-bb2c-905e60109595 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:18.258066 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 379/734] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:26:18.259349 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd108e64-3103-4cb5-ac6a-e9ca4a32b42d None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:18.326999 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc952bc3-ed3f-4852-bb2c-905e60109595 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:18.328469 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc952bc3-ed3f-4852-bb2c-905e60109595 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:18.346253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:18.379521 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd108e64-3103-4cb5-ac6a-e9ca4a32b42d None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:18.422023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efabfbe-1eee-4427-8713-0c949b1854dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:18.423187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efabfbe-1eee-4427-8713-0c949b1854dc None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:18.423473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efabfbe-1eee-4427-8713-0c949b1854dc None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:18.423871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efabfbe-1eee-4427-8713-0c949b1854dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:18.439170 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:18.439808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efabfbe-1eee-4427-8713-0c949b1854dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:18.450174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-3efabfbe-1eee-4427-8713-0c949b1854dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:18.450174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:18.450174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:18.450174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:18.450174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:18.450174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:18.450174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:18.450174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:18.450174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:18.450174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:18.450174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:18.450174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:18.450174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:18.450174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:18.450174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:18.450174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:18.452196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:18.452196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:18.452196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:18.452196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:18.452196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:18.452196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:18.452196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:18.452196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:18.452196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:18.452196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:18.452196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:18.452196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:18.452196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:18.452196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:18.452196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:18.452196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:18.453871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:18.453871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:18.453871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:18.453871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:18.453871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:18.453871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:18.453871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:18.453871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:18.453871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:18.453871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:18.453871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:18.453871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:18.454896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 356/735] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:26:17 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:26:18.454992 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 380/736] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:26:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:18.455065 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc952bc3-ed3f-4852-bb2c-905e60109595 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:18.455152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267ea6e-98a6-4446-b458-18f49036c194 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:18.455152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267ea6e-98a6-4446-b458-18f49036c194 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:18.455152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267ea6e-98a6-4446-b458-18f49036c194 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:18.455962 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 381/737] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:18.464896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267ea6e-98a6-4446-b458-18f49036c194 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:18.572249 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267ea6e-98a6-4446-b458-18f49036c194 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:18.577827 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc952bc3-ed3f-4852-bb2c-905e60109595 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:18.606642 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:18.611847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf777826-a83b-4301-8757-e8083b3eba14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:18.612734 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf777826-a83b-4301-8757-e8083b3eba14 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:18.613021 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf777826-a83b-4301-8757-e8083b3eba14 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:18.613300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf777826-a83b-4301-8757-e8083b3eba14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:18.617206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d667078-79c6-4923-91e7-24132e52dc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:18.618168 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d667078-79c6-4923-91e7-24132e52dc0a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:18.622908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d667078-79c6-4923-91e7-24132e52dc0a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:18.622908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d667078-79c6-4923-91e7-24132e52dc0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:18.623501 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf777826-a83b-4301-8757-e8083b3eba14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:18.624950 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf777826-a83b-4301-8757-e8083b3eba14 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:18.630690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d667078-79c6-4923-91e7-24132e52dc0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:18.639231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-7d667078-79c6-4923-91e7-24132e52dc0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:26:18.639231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:18.639231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:18.639231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:18.639231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:18.639231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:18.639231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:18.639231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:18.639231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:18.639231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:18.639231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:18.639231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:18.639231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:18.639231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:18.639231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:18.639231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:18.640436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:18.640436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:18.640436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:18.640436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:18.640436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:18.640436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:18.640436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:18.640436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:18.640436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:18.640436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:18.640436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:18.640436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:18.640436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:18.640436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:18.640436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:18.640436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:18.641448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:18.641448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:18.641448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:18.641448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:18.641448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:18.641448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:18.641448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:18.641448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:18.641448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:18.641448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:18.641448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:26:18.641448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:18.643006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-bf777826-a83b-4301-8757-e8083b3eba14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:18.643006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:18.643006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:18.643006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:18.643006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:18.643006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:18.643006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:18.643006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:18.643006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:18.643006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:18.643006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:18.643006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:18.643006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:18.643006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:18.643006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:18.644030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:18.644030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:18.644030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:18.644030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:18.644030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:18.644030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:18.644030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:18.644030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:18.644030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:18.644030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:18.644030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:18.644030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:18.644030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:18.644030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:18.644030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:18.644030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:18.644030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:18.645145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:18.645145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:18.645145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:18.645145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:18.650610 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 382/738] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:18.650610 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09f25e-ec0c-4878-8ad3-2a946aff5dfb None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:18.650610 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09f25e-ec0c-4878-8ad3-2a946aff5dfb None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:18.650610 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09f25e-ec0c-4878-8ad3-2a946aff5dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:18.651236 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 357/739] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:26:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:18.652204 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d667078-79c6-4923-91e7-24132e52dc0a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:18.668040 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-5c09f25e-ec0c-4878-8ad3-2a946aff5dfb None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:18.669294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-5c09f25e-ec0c-4878-8ad3-2a946aff5dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:18.711808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:18.742574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 383/740] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:26:18.742574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09f25e-ec0c-4878-8ad3-2a946aff5dfb None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:18.766206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d667078-79c6-4923-91e7-24132e52dc0a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:18.802174 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea10a72-214b-47cd-bddc-04dbf7c6359c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:18.803172 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea10a72-214b-47cd-bddc-04dbf7c6359c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:18.803602 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea10a72-214b-47cd-bddc-04dbf7c6359c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:18.804053 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea10a72-214b-47cd-bddc-04dbf7c6359c None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:18.804805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea10a72-214b-47cd-bddc-04dbf7c6359c None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:18.805111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea10a72-214b-47cd-bddc-04dbf7c6359c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:18.806477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea10a72-214b-47cd-bddc-04dbf7c6359c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:18.815449 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 358/741] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:26:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:18.821043 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea10a72-214b-47cd-bddc-04dbf7c6359c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:18.855682 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09f25e-ec0c-4878-8ad3-2a946aff5dfb None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:18.873608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:18.893937 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb7d2e-a3a9-4274-91aa-3f29e1187cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:18.894640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb7d2e-a3a9-4274-91aa-3f29e1187cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:18.894893 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb7d2e-a3a9-4274-91aa-3f29e1187cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:18.895172 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb7d2e-a3a9-4274-91aa-3f29e1187cf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:18.895893 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb7d2e-a3a9-4274-91aa-3f29e1187cf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:18.896206 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb7d2e-a3a9-4274-91aa-3f29e1187cf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:18.897587 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb7d2e-a3a9-4274-91aa-3f29e1187cf6 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:18.910312 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 384/742] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:18.911382 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb7d2e-a3a9-4274-91aa-3f29e1187cf6 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:18.932449 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea10a72-214b-47cd-bddc-04dbf7c6359c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:18.948646 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:18.971203 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0a093-27c9-4c71-8437-31dba822041c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:18.972354 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0a093-27c9-4c71-8437-31dba822041c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:18.972643 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0a093-27c9-4c71-8437-31dba822041c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:18.972925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0a093-27c9-4c71-8437-31dba822041c None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:18.973819 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0a093-27c9-4c71-8437-31dba822041c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:18.974134 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0a093-27c9-4c71-8437-31dba822041c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:18.975548 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0a093-27c9-4c71-8437-31dba822041c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:18.984684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 359/743] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:26:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:18.985466 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0a093-27c9-4c71-8437-31dba822041c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:19.022326 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb7d2e-a3a9-4274-91aa-3f29e1187cf6 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:19.060906 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4520c8-0178-424e-9905-ba81a483b0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:19.061570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4520c8-0178-424e-9905-ba81a483b0bc None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:19.061854 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4520c8-0178-424e-9905-ba81a483b0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:19.062129 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4520c8-0178-424e-9905-ba81a483b0bc None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:19.062964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4520c8-0178-424e-9905-ba81a483b0bc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:19.064340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4520c8-0178-424e-9905-ba81a483b0bc None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:19.094291 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0a093-27c9-4c71-8437-31dba822041c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:19.127149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6b1e8-c36d-4efd-8d34-fa186b88bec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:19.127945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6b1e8-c36d-4efd-8d34-fa186b88bec5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:19.128228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6b1e8-c36d-4efd-8d34-fa186b88bec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:19.128574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6b1e8-c36d-4efd-8d34-fa186b88bec5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:19.134037 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 385/744] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 23 11:26:18 2022] POST /identity/v3/services => generated 240 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:26:19.139611 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4520c8-0178-424e-9905-ba81a483b0bc None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:19.142679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6b1e8-c36d-4efd-8d34-fa186b88bec5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:19.144751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6b1e8-c36d-4efd-8d34-fa186b88bec5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:19.164797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-dfe6b1e8-c36d-4efd-8d34-fa186b88bec5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:19.164797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:19.164797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:19.164797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:19.164797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:19.164797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:19.164797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:19.164797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:19.164797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:19.164797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:19.164797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:19.164797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:19.164797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:19.164797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:19.164797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:19.165788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:19.165788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:19.165788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:19.165788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:19.165788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:19.165788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:19.165788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:19.165788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:19.165788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:19.165788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:19.165788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:19.165788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:19.165788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:19.165788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:19.165788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:19.165788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:19.165788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:19.166701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:19.166701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:19.166701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:19.166701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:19.169538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 360/745] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:19.170678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6b1e8-c36d-4efd-8d34-fa186b88bec5 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:19.192173 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:19.218986 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:19.294120 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6b1e8-c36d-4efd-8d34-fa186b88bec5 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:19.299335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4520c8-0178-424e-9905-ba81a483b0bc None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:19.333898 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa34b7-8c9e-45d3-afd5-d81ba9e62d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:19.334722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa34b7-8c9e-45d3-afd5-d81ba9e62d26 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:19.335008 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa34b7-8c9e-45d3-afd5-d81ba9e62d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:19.335285 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa34b7-8c9e-45d3-afd5-d81ba9e62d26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:19.343047 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd691b30-edbe-4625-a688-66783a1a59eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:19.343864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd691b30-edbe-4625-a688-66783a1a59eb None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:19.344147 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd691b30-edbe-4625-a688-66783a1a59eb None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:19.344416 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd691b30-edbe-4625-a688-66783a1a59eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:19.346591 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa34b7-8c9e-45d3-afd5-d81ba9e62d26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:19.348357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa34b7-8c9e-45d3-afd5-d81ba9e62d26 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:19.353631 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd691b30-edbe-4625-a688-66783a1a59eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:19.355107 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd691b30-edbe-4625-a688-66783a1a59eb None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:19.368879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-9caa34b7-8c9e-45d3-afd5-d81ba9e62d26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:19.368879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:19.368879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:19.368879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:19.368879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:19.368879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:19.368879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:19.368879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:19.368879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:19.368879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:19.368879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:19.368879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:19.368879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:19.368879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:19.368879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:19.369793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:19.369793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:19.369793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:19.369793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:19.369793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:19.369793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:19.369793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:19.369793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:19.369793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:19.369793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:19.369793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:19.369793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:19.369793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:19.369793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:19.369793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:19.369793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:19.369793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:19.370702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:19.370702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:19.370702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:19.370702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:19.370949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-cd691b30-edbe-4625-a688-66783a1a59eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:19.370949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:19.370949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:19.370949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:19.370949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:19.370949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:19.370949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:19.370949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:19.370949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:19.370949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:19.370949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:19.370949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:19.370949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:19.370949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:19.370949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:19.372345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:19.372345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:19.372345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:19.372345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:19.372345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:19.372345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:19.372345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:19.372345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:19.372345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:19.372345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:19.372345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:19.372345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:19.372345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:19.372345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:19.372345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:19.372345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:19.372345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:19.373323 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:19.373323 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:19.373323 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:19.373323 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:19.373323 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:19.373323 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:19.373323 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:19.373323 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:19.373323 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 386/746] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:19.373323 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd691b30-edbe-4625-a688-66783a1a59eb None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:19.376498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 361/747] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:19.380333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa34b7-8c9e-45d3-afd5-d81ba9e62d26 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:19.472870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:19.492464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:19.503676 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa34b7-8c9e-45d3-afd5-d81ba9e62d26 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:19.507288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd691b30-edbe-4625-a688-66783a1a59eb None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:19.542142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ef8d3-e1fb-49ab-b6cd-200a3eb44d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:19.542727 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ef8d3-e1fb-49ab-b6cd-200a3eb44d16 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:19.543013 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ef8d3-e1fb-49ab-b6cd-200a3eb44d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:19.543283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ef8d3-e1fb-49ab-b6cd-200a3eb44d16 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:19.544115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ef8d3-e1fb-49ab-b6cd-200a3eb44d16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:19.544428 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ef8d3-e1fb-49ab-b6cd-200a3eb44d16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:19.545853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ef8d3-e1fb-49ab-b6cd-200a3eb44d16 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:19.556562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc03f79-ec72-4887-91ec-12ef37e9ed47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:19.557520 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc03f79-ec72-4887-91ec-12ef37e9ed47 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:19.558012 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc03f79-ec72-4887-91ec-12ef37e9ed47 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:19.558492 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc03f79-ec72-4887-91ec-12ef37e9ed47 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:19.558713 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 362/748] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:19.560125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ef8d3-e1fb-49ab-b6cd-200a3eb44d16 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:19.560570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc03f79-ec72-4887-91ec-12ef37e9ed47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:19.561094 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc03f79-ec72-4887-91ec-12ef37e9ed47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:19.562897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc03f79-ec72-4887-91ec-12ef37e9ed47 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:19.580390 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 387/749] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:19.581560 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc03f79-ec72-4887-91ec-12ef37e9ed47 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:19.674220 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ef8d3-e1fb-49ab-b6cd-200a3eb44d16 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:19.711655 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7026d-575c-475e-9728-da570bf0db5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:19.712311 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7026d-575c-475e-9728-da570bf0db5a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:19.712590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7026d-575c-475e-9728-da570bf0db5a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:19.712861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7026d-575c-475e-9728-da570bf0db5a None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:19.713634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa7026d-575c-475e-9728-da570bf0db5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:19.713982 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa7026d-575c-475e-9728-da570bf0db5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:19.715996 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa7026d-575c-475e-9728-da570bf0db5a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:19.717900 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc03f79-ec72-4887-91ec-12ef37e9ed47 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:19.725208 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:19.734479 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 363/750] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:19.736167 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7026d-575c-475e-9728-da570bf0db5a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:19.762440 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60004392-3d80-4556-a4ac-997b1dd8b9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:19.763709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60004392-3d80-4556-a4ac-997b1dd8b9cf None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:19.764561 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60004392-3d80-4556-a4ac-997b1dd8b9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:19.765497 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60004392-3d80-4556-a4ac-997b1dd8b9cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:19.766757 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60004392-3d80-4556-a4ac-997b1dd8b9cf None admin] RBAC: Adding query filter params (role.id=c9e006de0d6c4812ab29d778e9c8a830, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=450b1fad96604fb09d0705895c6d3da4) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:19.767579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60004392-3d80-4556-a4ac-997b1dd8b9cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:19.769447 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60004392-3d80-4556-a4ac-997b1dd8b9cf None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:19.777028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 388/751] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:26:19 2022] GET /identity/v3/role_assignments?user.id=450b1fad96604fb09d0705895c6d3da4&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=c9e006de0d6c4812ab29d778e9c8a830 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:19.778793 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60004392-3d80-4556-a4ac-997b1dd8b9cf None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:19.810781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:19.905575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7026d-575c-475e-9728-da570bf0db5a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:19.959881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0140877-973b-4148-bd95-dcaa9116bdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:19.960709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0140877-973b-4148-bd95-dcaa9116bdff None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:19.960995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0140877-973b-4148-bd95-dcaa9116bdff None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:19.961273 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0140877-973b-4148-bd95-dcaa9116bdff None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:19.968079 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60004392-3d80-4556-a4ac-997b1dd8b9cf None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:19.969664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0140877-973b-4148-bd95-dcaa9116bdff None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:19.971672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0140877-973b-4148-bd95-dcaa9116bdff None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:20.005901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-f0140877-973b-4148-bd95-dcaa9116bdff None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:26:20.005901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:20.005901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:20.005901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:20.005901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:20.005901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:20.005901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:20.005901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:20.005901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:20.005901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:20.005901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:20.005901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:20.005901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:20.005901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:20.005901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:20.005901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:20.006939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:20.006939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:20.006939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:20.006939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:20.006939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:20.006939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:20.006939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:20.006939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:20.006939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:20.006939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:20.006939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:20.006939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:20.006939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:20.006939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:20.006939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:20.006939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:20.006939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:20.006939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:20.007882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:20.007882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:20.007882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:20.007882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:20.007882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:20.007882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:20.007882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:20.007882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:20.007882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:26:20.007882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:20.007882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 364/752] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jun 23 11:26:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:20.010205 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a2c61-baba-43eb-b42c-03fdd88b40af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:20.010205 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a2c61-baba-43eb-b42c-03fdd88b40af None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:20.010205 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a2c61-baba-43eb-b42c-03fdd88b40af None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:20.010637 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a2c61-baba-43eb-b42c-03fdd88b40af None admin] PATH_INFO: `/v3/users/nova` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:20.016266 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0140877-973b-4148-bd95-dcaa9116bdff None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:20.036649 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a2c61-baba-43eb-b42c-03fdd88b40af None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:20.038417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a2c61-baba-43eb-b42c-03fdd88b40af None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:20.072139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-c16a2c61-baba-43eb-b42c-03fdd88b40af None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:26:20.072139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:20.072139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:20.072139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:20.072139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:20.072139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:20.072139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:20.072139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:20.072139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:20.072139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:20.072139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:20.072139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:20.072139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:20.072139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:20.072139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:20.072139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:20.073367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:20.073367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:20.073367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:20.073367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:20.073367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:20.073367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:20.073367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:20.073367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:20.073367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:20.073367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:20.073367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:20.073367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:20.073367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:20.073367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:20.073367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:20.073367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:20.073367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:20.073367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:20.074573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:20.074573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:20.074573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:20.074573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:20.074573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:20.074573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:20.074573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:20.074573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:20.074573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:26:20.074573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:20.078873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 389/753] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jun 23 11:26:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:20.085448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a2c61-baba-43eb-b42c-03fdd88b40af None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:20.147105 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:20.201424 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0140877-973b-4148-bd95-dcaa9116bdff None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:20.213912 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:20.240970 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82bb47-bd14-4483-90a3-c4476aa28868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:20.241643 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82bb47-bd14-4483-90a3-c4476aa28868 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:20.241930 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82bb47-bd14-4483-90a3-c4476aa28868 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:20.242208 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82bb47-bd14-4483-90a3-c4476aa28868 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:20.243841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82bb47-bd14-4483-90a3-c4476aa28868 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:20.244159 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82bb47-bd14-4483-90a3-c4476aa28868 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:20.245592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82bb47-bd14-4483-90a3-c4476aa28868 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:20.249511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a2c61-baba-43eb-b42c-03fdd88b40af None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:20.273098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 365/754] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Jun 23 11:26:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:20.275441 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c420d25-a69d-4978-ae07-2b1f15b1be53 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:20.275925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c420d25-a69d-4978-ae07-2b1f15b1be53 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:20.276623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c420d25-a69d-4978-ae07-2b1f15b1be53 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:20.283182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 366/755] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:20.283551 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c420d25-a69d-4978-ae07-2b1f15b1be53 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:20.292587 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f40efc-1f27-4150-88ed-7967edf99642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:20.293260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f40efc-1f27-4150-88ed-7967edf99642 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:20.293573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f40efc-1f27-4150-88ed-7967edf99642 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:20.294195 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f40efc-1f27-4150-88ed-7967edf99642 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:20.294801 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f40efc-1f27-4150-88ed-7967edf99642 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:20.295114 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f40efc-1f27-4150-88ed-7967edf99642 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:20.296696 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f40efc-1f27-4150-88ed-7967edf99642 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:20.317010 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 390/756] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jun 23 11:26:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:20.318740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff328ec7-e0fc-4036-93d3-b0c24b1d8630 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:20.319033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff328ec7-e0fc-4036-93d3-b0c24b1d8630 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:20.319308 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff328ec7-e0fc-4036-93d3-b0c24b1d8630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:20.337595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-ff328ec7-e0fc-4036-93d3-b0c24b1d8630 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:20.339100 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-ff328ec7-e0fc-4036-93d3-b0c24b1d8630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:20.419027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c420d25-a69d-4978-ae07-2b1f15b1be53 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:20.435402 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 391/757] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:20.435402 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff328ec7-e0fc-4036-93d3-b0c24b1d8630 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:20.453755 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:20.477437 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655cf267-9852-4866-aab2-d76dc9941b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:20.478700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cf267-9852-4866-aab2-d76dc9941b01 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:20.479460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cf267-9852-4866-aab2-d76dc9941b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:20.480335 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cf267-9852-4866-aab2-d76dc9941b01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:20.523385 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:20.567674 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff328ec7-e0fc-4036-93d3-b0c24b1d8630 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:20.573001 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655cf267-9852-4866-aab2-d76dc9941b01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:20.574799 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655cf267-9852-4866-aab2-d76dc9941b01 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:20.604851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e4d3d-4867-44b7-be0e-249f098d6547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:20.605621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e4d3d-4867-44b7-be0e-249f098d6547 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:20.606541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e4d3d-4867-44b7-be0e-249f098d6547 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:20.606830 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e4d3d-4867-44b7-be0e-249f098d6547 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:20.618461 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e4d3d-4867-44b7-be0e-249f098d6547 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:20.620051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e4d3d-4867-44b7-be0e-249f098d6547 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:20.638869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-bb7e4d3d-4867-44b7-be0e-249f098d6547 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:20.638869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:20.638869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:20.638869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:20.638869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:20.638869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:20.638869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:20.638869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:20.638869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:20.638869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:20.638869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:20.638869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:20.638869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:20.638869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:20.638869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:20.639806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:20.639806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:20.639806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:20.639806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:20.639806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:20.639806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:20.639806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:20.639806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:20.639806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:20.639806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:20.639806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:20.639806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:20.639806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:20.639806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:20.639806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:20.639806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:20.639806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:20.640689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:20.640689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:20.640689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:20.640689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:20.640689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 392/758] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:20.641400 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e4d3d-4867-44b7-be0e-249f098d6547 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:20.700829 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 367/759] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:26:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:26:20.701759 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655cf267-9852-4866-aab2-d76dc9941b01 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:20.721095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:20.769144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e4d3d-4867-44b7-be0e-249f098d6547 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:20.786716 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:20.807732 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf357c5a-0a71-4095-ad48-5664bd91bb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:20.808546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf357c5a-0a71-4095-ad48-5664bd91bb1a None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:20.808835 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf357c5a-0a71-4095-ad48-5664bd91bb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:20.809141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf357c5a-0a71-4095-ad48-5664bd91bb1a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:20.809800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf357c5a-0a71-4095-ad48-5664bd91bb1a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:20.811299 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf357c5a-0a71-4095-ad48-5664bd91bb1a None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:20.820854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655cf267-9852-4866-aab2-d76dc9941b01 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:20.827226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-bf357c5a-0a71-4095-ad48-5664bd91bb1a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 11:26:20.827226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:20.827226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:20.827226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:20.827226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:20.827226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:20.827226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:20.827226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:20.827226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 11:26:20.827226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:20.827226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:20.827226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:20.827226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:20.827226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:20.827226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:20.827226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:20.828168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:20.828168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:20.828168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:20.828168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:20.828168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:20.828168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:20.828168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:20.828168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:20.828168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:20.828168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:20.828168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:20.828168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:20.828168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:20.828168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:20.828168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:20.828168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:20.829053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:20.829053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:20.829053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:20.829053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:20.829053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:20.829053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:20.829053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:20.829053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:20.829053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:20.829053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:20.829053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:20.829053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:20.829053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:20.829053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 11:26:20.829053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:20.832965 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 393/760] 10.222.0.13 () {56 vars in 1129 bytes} [Thu Jun 23 11:26:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:20.833809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf357c5a-0a71-4095-ad48-5664bd91bb1a None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:20.865537 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d748b-cd08-4643-8ea3-073f11fb90d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:20.866716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d748b-cd08-4643-8ea3-073f11fb90d6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:20.867271 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d748b-cd08-4643-8ea3-073f11fb90d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:20.867743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d748b-cd08-4643-8ea3-073f11fb90d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:20.868661 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d748b-cd08-4643-8ea3-073f11fb90d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:20.869166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d748b-cd08-4643-8ea3-073f11fb90d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:20.871051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d748b-cd08-4643-8ea3-073f11fb90d6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:20.884258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 368/761] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:20.885373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d748b-cd08-4643-8ea3-073f11fb90d6 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:20.939374 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf357c5a-0a71-4095-ad48-5664bd91bb1a None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:20.976015 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dca68a-c519-49f5-9a8a-31f4d1415b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:20.977037 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:20.977473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dca68a-c519-49f5-9a8a-31f4d1415b1d None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:20.978594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dca68a-c519-49f5-9a8a-31f4d1415b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:20.979029 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dca68a-c519-49f5-9a8a-31f4d1415b1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:20.991006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dca68a-c519-49f5-9a8a-31f4d1415b1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:20.992764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dca68a-c519-49f5-9a8a-31f4d1415b1d None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:21.002196 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d748b-cd08-4643-8ea3-073f11fb90d6 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:21.014691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-14dca68a-c519-49f5-9a8a-31f4d1415b1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:21.014691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:21.014691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:21.014691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:21.014691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:21.014691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:21.014691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:21.014691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:21.014691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:21.014691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:21.014691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:21.014691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:21.014691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:21.014691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:21.014691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:21.015822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:21.015822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:21.015822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:21.015822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:21.015822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:21.015822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:21.015822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:21.015822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:21.015822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:21.015822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:21.015822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:21.015822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:21.015822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:21.015822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:21.015822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:21.015822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:21.015822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:21.016885 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:21.016885 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:21.016885 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:21.016885 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:21.018505 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 394/762] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:21.019457 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dca68a-c519-49f5-9a8a-31f4d1415b1d None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:21.039952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee39f77-6a46-451f-ab75-321d956498af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:21.040556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee39f77-6a46-451f-ab75-321d956498af None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:21.040841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee39f77-6a46-451f-ab75-321d956498af None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:21.041128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee39f77-6a46-451f-ab75-321d956498af None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:21.041947 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee39f77-6a46-451f-ab75-321d956498af None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:21.042266 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee39f77-6a46-451f-ab75-321d956498af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:21.044228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee39f77-6a46-451f-ab75-321d956498af None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:21.053913 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 369/763] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Jun 23 11:26:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:21.053913 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee39f77-6a46-451f-ab75-321d956498af None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:21.060560 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:21.135612 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dca68a-c519-49f5-9a8a-31f4d1415b1d None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:21.159912 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee39f77-6a46-451f-ab75-321d956498af None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:21.173987 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd255b6-0174-42d5-ae6f-a7e845a3491f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:21.175701 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd255b6-0174-42d5-ae6f-a7e845a3491f None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:21.176173 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd255b6-0174-42d5-ae6f-a7e845a3491f None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:21.176839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd255b6-0174-42d5-ae6f-a7e845a3491f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:21.186715 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd255b6-0174-42d5-ae6f-a7e845a3491f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:21.188972 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd255b6-0174-42d5-ae6f-a7e845a3491f None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:21.196874 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165f66c-0d9e-4c9f-8c96-b541761783ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:21.197610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165f66c-0d9e-4c9f-8c96-b541761783ec None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:21.197969 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165f66c-0d9e-4c9f-8c96-b541761783ec None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:21.198334 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165f66c-0d9e-4c9f-8c96-b541761783ec None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:21.199158 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165f66c-0d9e-4c9f-8c96-b541761783ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:21.199564 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165f66c-0d9e-4c9f-8c96-b541761783ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:21.200994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165f66c-0d9e-4c9f-8c96-b541761783ec None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:21.206162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-ebd255b6-0174-42d5-ae6f-a7e845a3491f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:21.206162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:21.206162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:21.206162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:21.206162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:21.206162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:21.206162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:21.206162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:21.206162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:21.206162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:21.206162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:21.206162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:21.206162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:21.206162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:21.206162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:21.207224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:21.207224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:21.207224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:21.207224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:21.207224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:21.207224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:21.207224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:21.207224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:21.207224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:21.207224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:21.207224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:21.207224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:21.207224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:21.207224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:21.207224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:21.207224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:21.207224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:21.208325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:21.208325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:21.208325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:21.208325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:21.208325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:21.208325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:21.208325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:21.208325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:21.210987 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 395/764] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:21.212510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd255b6-0174-42d5-ae6f-a7e845a3491f None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:21.220277 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 370/765] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:21.221251 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165f66c-0d9e-4c9f-8c96-b541761783ec None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:21.224449 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:21.326294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:21.336671 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd255b6-0174-42d5-ae6f-a7e845a3491f None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:21.342902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165f66c-0d9e-4c9f-8c96-b541761783ec None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:21.383605 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4ee0b-3f66-432d-a200-b948dea7f389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:21.384582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4ee0b-3f66-432d-a200-b948dea7f389 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:21.385068 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4ee0b-3f66-432d-a200-b948dea7f389 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:21.385519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4ee0b-3f66-432d-a200-b948dea7f389 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:21.385623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de439a-3394-471d-89ea-357dd4a2c3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:21.386294 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de439a-3394-471d-89ea-357dd4a2c3a0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:21.386616 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de439a-3394-471d-89ea-357dd4a2c3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:21.386899 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de439a-3394-471d-89ea-357dd4a2c3a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:21.388002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4ee0b-3f66-432d-a200-b948dea7f389 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:21.388328 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de439a-3394-471d-89ea-357dd4a2c3a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:21.388646 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de439a-3394-471d-89ea-357dd4a2c3a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:21.388833 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4ee0b-3f66-432d-a200-b948dea7f389 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:21.390131 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de439a-3394-471d-89ea-357dd4a2c3a0 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:21.390662 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4ee0b-3f66-432d-a200-b948dea7f389 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:21.397119 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 396/766] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Jun 23 11:26:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:21.398213 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4ee0b-3f66-432d-a200-b948dea7f389 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:21.409409 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 371/767] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:21.411000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6005b7d-3da2-409d-89e0-cb80078fea64 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:21.411298 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6005b7d-3da2-409d-89e0-cb80078fea64 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:21.411785 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6005b7d-3da2-409d-89e0-cb80078fea64 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:21.413670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 372/768] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:21.414381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6005b7d-3da2-409d-89e0-cb80078fea64 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:21.483007 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:21.513637 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4ee0b-3f66-432d-a200-b948dea7f389 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:21.540788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6005b7d-3da2-409d-89e0-cb80078fea64 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:21.550513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf88e0-8140-47a1-a1b3-d566f1f9de00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:21.551332 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf88e0-8140-47a1-a1b3-d566f1f9de00 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:21.551619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf88e0-8140-47a1-a1b3-d566f1f9de00 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:21.551898 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf88e0-8140-47a1-a1b3-d566f1f9de00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:21.558976 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cf88e0-8140-47a1-a1b3-d566f1f9de00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:21.560446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cf88e0-8140-47a1-a1b3-d566f1f9de00 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:21.575164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-a5cf88e0-8140-47a1-a1b3-d566f1f9de00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:21.575164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:21.575164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:21.575164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:21.575164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:21.575164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:21.575164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:21.575164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:21.575164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:21.575164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:21.575164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:21.575164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:21.575164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:21.575164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:21.575164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:21.576121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:21.576121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:21.576121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:21.576121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:21.576121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:21.576121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:21.576121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:21.576121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:21.576121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:21.576121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:21.576121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:21.576121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:21.576121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:21.576121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:21.576121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:21.576121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:21.576121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:21.577083 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:21.577083 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:21.577083 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:21.577083 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:21.577083 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:21.577083 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:21.577083 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:21.577083 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:21.577083 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 397/769] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:21.577083 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf88e0-8140-47a1-a1b3-d566f1f9de00 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:21.580150 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6a88c-f0e3-4c0f-be15-bb1c92bbfe55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:21.580816 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6a88c-f0e3-4c0f-be15-bb1c92bbfe55 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:21.581101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6a88c-f0e3-4c0f-be15-bb1c92bbfe55 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:21.581373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6a88c-f0e3-4c0f-be15-bb1c92bbfe55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:21.582163 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6a88c-f0e3-4c0f-be15-bb1c92bbfe55 None admin] RBAC: Adding query filter params (interface=public, service_id=c0d0e7c1756b428592b58e8b242d2275, region_id=RegionOne) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:21.582649 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6a88c-f0e3-4c0f-be15-bb1c92bbfe55 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:21.583881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6a88c-f0e3-4c0f-be15-bb1c92bbfe55 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:21.594253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 373/770] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jun 23 11:26:21 2022] GET /identity/v3/endpoints?service_id=c0d0e7c1756b428592b58e8b242d2275&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:21.596210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cde413-1a84-4eca-ab18-8205506d2d5a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:21.597043 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cde413-1a84-4eca-ab18-8205506d2d5a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:21.597519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cde413-1a84-4eca-ab18-8205506d2d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:21.599563 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:21.618019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-51cde413-1a84-4eca-ab18-8205506d2d5a None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:21.619237 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-51cde413-1a84-4eca-ab18-8205506d2d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:21.692773 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf88e0-8140-47a1-a1b3-d566f1f9de00 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:21.706556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 374/771] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:21.706556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cde413-1a84-4eca-ab18-8205506d2d5a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:21.732812 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c04003-0e87-480c-a993-a0a68f7b7deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:21.733698 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c04003-0e87-480c-a993-a0a68f7b7deb None admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:21.733983 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c04003-0e87-480c-a993-a0a68f7b7deb None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:21.734257 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c04003-0e87-480c-a993-a0a68f7b7deb None admin] PATH_INFO: `/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/161c5ed1e47e450290a1f428ee100b68/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:21.736271 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c04003-0e87-480c-a993-a0a68f7b7deb None admin] RBAC: Authorizing `identity:create_grant(project_id=886c2939a14b48febc2e2fe639c8f1e4, user_id=161c5ed1e47e450290a1f428ee100b68, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:21.737712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c04003-0e87-480c-a993-a0a68f7b7deb None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:21.768138 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:21.811436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cde413-1a84-4eca-ab18-8205506d2d5a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:21.815148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 398/772] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:26:21 2022] PUT /identity/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/161c5ed1e47e450290a1f428ee100b68/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:26:21.823681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c04003-0e87-480c-a993-a0a68f7b7deb None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:21.851822 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9fa29-c38a-4a0a-a316-ade1cc07d5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:21.852595 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9fa29-c38a-4a0a-a316-ade1cc07d5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:21.852985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9fa29-c38a-4a0a-a316-ade1cc07d5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:21.853268 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9fa29-c38a-4a0a-a316-ade1cc07d5f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:21.854184 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9fa29-c38a-4a0a-a316-ade1cc07d5f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:21.854501 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9fa29-c38a-4a0a-a316-ade1cc07d5f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:21.856676 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9fa29-c38a-4a0a-a316-ade1cc07d5f9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:21.877714 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 375/773] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:21.877714 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9fa29-c38a-4a0a-a316-ade1cc07d5f9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:21.954792 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:21.985108 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c04003-0e87-480c-a993-a0a68f7b7deb None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:22.010668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9fa29-c38a-4a0a-a316-ade1cc07d5f9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:22.026395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d16878-9108-4478-865c-c65767a1edf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:22.026822 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:22.029068 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d16878-9108-4478-865c-c65767a1edf3 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:22.029524 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d16878-9108-4478-865c-c65767a1edf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:22.029942 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d16878-9108-4478-865c-c65767a1edf3 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:22.030797 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d16878-9108-4478-865c-c65767a1edf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:22.033035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d16878-9108-4478-865c-c65767a1edf3 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:22.040975 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 399/774] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jun 23 11:26:21 2022] GET /identity/v3/services => generated 772 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:22.042255 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d16878-9108-4478-865c-c65767a1edf3 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:22.045918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c5627-2eda-42a8-a9d4-fca73c3ae808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:22.047315 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c5627-2eda-42a8-a9d4-fca73c3ae808 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:22.047708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c5627-2eda-42a8-a9d4-fca73c3ae808 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:22.048089 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c5627-2eda-42a8-a9d4-fca73c3ae808 None admin] PATH_INFO: `/v3/services/network` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:22.050007 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3c5627-2eda-42a8-a9d4-fca73c3ae808 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:22.051586 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3c5627-2eda-42a8-a9d4-fca73c3ae808 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:22.069922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-7b3c5627-2eda-42a8-a9d4-fca73c3ae808 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 11:26:22.069922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:22.069922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:22.069922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:22.069922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:22.069922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:22.069922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:22.069922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:22.069922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 11:26:22.069922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:22.069922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:22.069922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:22.069922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:22.069922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:22.069922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:22.069922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:22.069922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:22.071111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:22.071111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:22.071111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:22.071111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:22.071111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:22.071111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:22.071111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:22.071111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:22.071111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:22.071111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:22.071111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:22.071111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:22.071111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:22.071111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:22.071111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:22.071111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:22.071951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:22.071951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:22.071951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:22.071951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:22.071951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:22.071951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:22.071951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:22.071951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:22.071951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:22.071951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:22.071951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:22.071951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:22.071951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 11:26:22.071951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:22.072993 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 376/775] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:21 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:22.074079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846891f7-57ba-4302-8524-b9d36b9bb2b6 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:22.074443 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846891f7-57ba-4302-8524-b9d36b9bb2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:22.074962 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846891f7-57ba-4302-8524-b9d36b9bb2b6 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:22.076440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 377/776] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:22.079535 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846891f7-57ba-4302-8524-b9d36b9bb2b6 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:22.178438 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d16878-9108-4478-865c-c65767a1edf3 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:22.200964 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846891f7-57ba-4302-8524-b9d36b9bb2b6 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:22.221415 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a39a7-a9ae-43a8-82b7-755168a3e9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:22.222077 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a39a7-a9ae-43a8-82b7-755168a3e9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:22.223033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a39a7-a9ae-43a8-82b7-755168a3e9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:22.223033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a39a7-a9ae-43a8-82b7-755168a3e9d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:22.223464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a39a7-a9ae-43a8-82b7-755168a3e9d9 None admin] RBAC: Adding query filter params (role.id=13708a4819a94b94b47ba5994aaa9310, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=030614ef56f143f8be6fcc0445ec9568) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:22.223773 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a39a7-a9ae-43a8-82b7-755168a3e9d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:22.225205 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a39a7-a9ae-43a8-82b7-755168a3e9d9 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:22.232622 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 400/777] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:26:22 2022] GET /identity/v3/role_assignments?user.id=030614ef56f143f8be6fcc0445ec9568&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=13708a4819a94b94b47ba5994aaa9310 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:22.234022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a184b74-12e2-484a-8df7-69dcf4c69757 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:22.234134 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a184b74-12e2-484a-8df7-69dcf4c69757 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:22.234434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a184b74-12e2-484a-8df7-69dcf4c69757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:22.239586 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:22.253913 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066b046-c3fb-4d74-bd44-3906a4dc2a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:22.254140 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-3a184b74-12e2-484a-8df7-69dcf4c69757 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:22.255197 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-3a184b74-12e2-484a-8df7-69dcf4c69757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:22.255452 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066b046-c3fb-4d74-bd44-3906a4dc2a4f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:22.255934 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066b046-c3fb-4d74-bd44-3906a4dc2a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:22.256465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066b046-c3fb-4d74-bd44-3906a4dc2a4f None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:22.257731 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066b046-c3fb-4d74-bd44-3906a4dc2a4f None admin] RBAC: Adding query filter params (name=network) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:22.259331 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066b046-c3fb-4d74-bd44-3906a4dc2a4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:22.261079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066b046-c3fb-4d74-bd44-3906a4dc2a4f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:22.269467 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 378/778] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 23 11:26:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:22.275469 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066b046-c3fb-4d74-bd44-3906a4dc2a4f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:22.349449 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:22.395782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 401/779] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:22.431824 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a184b74-12e2-484a-8df7-69dcf4c69757 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:22.441610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066b046-c3fb-4d74-bd44-3906a4dc2a4f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:22.489143 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4547c4-e812-44d9-9f0b-d67e0802eb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:22.490268 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4547c4-e812-44d9-9f0b-d67e0802eb40 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:22.490778 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4547c4-e812-44d9-9f0b-d67e0802eb40 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:22.491385 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4547c4-e812-44d9-9f0b-d67e0802eb40 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:22.492293 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4547c4-e812-44d9-9f0b-d67e0802eb40 None admin] RBAC: Adding query filter params (type=network) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:22.492802 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4547c4-e812-44d9-9f0b-d67e0802eb40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:22.494580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4547c4-e812-44d9-9f0b-d67e0802eb40 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:22.503900 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 379/780] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 23 11:26:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:22.509531 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4547c4-e812-44d9-9f0b-d67e0802eb40 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:22.593357 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a184b74-12e2-484a-8df7-69dcf4c69757 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:22.651137 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:22.661948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b105aa7-df28-4e98-87a6-c78847c08ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:22.663133 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b105aa7-df28-4e98-87a6-c78847c08ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:22.663722 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b105aa7-df28-4e98-87a6-c78847c08ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:22.664200 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b105aa7-df28-4e98-87a6-c78847c08ca5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:22.685019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b105aa7-df28-4e98-87a6-c78847c08ca5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:22.698828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-3b105aa7-df28-4e98-87a6-c78847c08ca5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:26:22.698828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:22.698828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:22.698828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:22.698828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:22.698828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:22.698828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:22.698828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:22.698828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:22.698828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:22.698828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:22.698828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:22.698828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:22.698828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:22.698828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:22.698828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:22.700569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:22.700569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:22.700569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:22.700569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:22.700569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:22.700569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:22.700569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:22.700569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:22.700569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:22.700569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:22.700569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:22.700569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:22.700569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:22.700569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:22.700569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:22.700569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:22.701405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:22.701405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:22.701405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:22.701405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:22.701405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:22.701405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:22.701405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:22.701405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:22.701405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:22.701405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:22.701405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:26:22.701405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:22.710555 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4547c4-e812-44d9-9f0b-d67e0802eb40 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:22.711430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 402/781] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:26:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:22.712995 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b105aa7-df28-4e98-87a6-c78847c08ca5 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:22.714405 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:22.740655 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72fa392-a559-4cff-967d-3f817b9c9be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:22.749627 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fa392-a559-4cff-967d-3f817b9c9be6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:22.749627 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fa392-a559-4cff-967d-3f817b9c9be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:22.749627 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fa392-a559-4cff-967d-3f817b9c9be6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:22.749627 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72fa392-a559-4cff-967d-3f817b9c9be6 None admin] RBAC: Adding query filter params (interface=public, service_id=4c64b18fa9014b24b1eb154b95aba8d1, region_id=RegionOne) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:22.749627 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72fa392-a559-4cff-967d-3f817b9c9be6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:22.749627 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72fa392-a559-4cff-967d-3f817b9c9be6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:22.751430 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 380/782] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jun 23 11:26:22 2022] GET /identity/v3/endpoints?service_id=4c64b18fa9014b24b1eb154b95aba8d1&interface=public®ion_id=RegionOne => generated 194 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:22.756679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72fa392-a559-4cff-967d-3f817b9c9be6 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:22.868891 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72fa392-a559-4cff-967d-3f817b9c9be6 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:22.908680 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b105aa7-df28-4e98-87a6-c78847c08ca5 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:22.909096 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476974a-e56d-49ab-90c1-484e5972d175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:22.909771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476974a-e56d-49ab-90c1-484e5972d175 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:22.910054 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476974a-e56d-49ab-90c1-484e5972d175 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:22.910492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476974a-e56d-49ab-90c1-484e5972d175 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:22.911210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476974a-e56d-49ab-90c1-484e5972d175 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:22.912590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476974a-e56d-49ab-90c1-484e5972d175 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:22.919691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 381/783] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jun 23 11:26:22 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:22.978283 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e052ef-5231-4330-b6f9-98cb33f14076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:22.978905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e052ef-5231-4330-b6f9-98cb33f14076 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:22.979186 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e052ef-5231-4330-b6f9-98cb33f14076 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:22.979456 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e052ef-5231-4330-b6f9-98cb33f14076 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:22.980209 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e052ef-5231-4330-b6f9-98cb33f14076 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:22.980514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e052ef-5231-4330-b6f9-98cb33f14076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:22.982450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e052ef-5231-4330-b6f9-98cb33f14076 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:22.991747 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 403/784] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:26:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:22.997819 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476974a-e56d-49ab-90c1-484e5972d175 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:23.154450 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:23.221290 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476974a-e56d-49ab-90c1-484e5972d175 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:23.256392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79974e17-9fa5-4fa4-8c99-a4d68bff5308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:23.257109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79974e17-9fa5-4fa4-8c99-a4d68bff5308 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:23.257396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79974e17-9fa5-4fa4-8c99-a4d68bff5308 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:23.257667 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79974e17-9fa5-4fa4-8c99-a4d68bff5308 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:23.269188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79974e17-9fa5-4fa4-8c99-a4d68bff5308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:23.270659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79974e17-9fa5-4fa4-8c99-a4d68bff5308 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:23.288021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-79974e17-9fa5-4fa4-8c99-a4d68bff5308 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:23.288021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:23.288021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:23.288021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:23.288021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:23.288021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:23.288021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:23.288021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:23.288021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:23.288021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:23.288021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:23.288021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:23.288021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:23.288021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:23.288021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:23.288908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:23.288908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:23.288908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:23.288908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:23.288908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:23.288908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:23.288908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:23.288908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:23.288908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:23.288908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:23.288908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:23.288908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:23.288908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:23.288908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:23.288908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:23.288908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:23.288908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:23.290009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:23.290009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:23.290009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:23.290009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:23.290009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 382/785] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:23.294944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e052ef-5231-4330-b6f9-98cb33f14076 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:23.354337 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:23.406337 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e052ef-5231-4330-b6f9-98cb33f14076 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:23.452870 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6d97c-ecc7-46f1-86a5-f1601753ae6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:23.453542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6d97c-ecc7-46f1-86a5-f1601753ae6b None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:23.453828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6d97c-ecc7-46f1-86a5-f1601753ae6b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:23.454106 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6d97c-ecc7-46f1-86a5-f1601753ae6b None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:23.454874 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce6d97c-ecc7-46f1-86a5-f1601753ae6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:23.455188 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce6d97c-ecc7-46f1-86a5-f1601753ae6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:23.456559 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce6d97c-ecc7-46f1-86a5-f1601753ae6b None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:23.469508 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 404/786] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:23.474745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79974e17-9fa5-4fa4-8c99-a4d68bff5308 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:23.592403 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:23.604276 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79974e17-9fa5-4fa4-8c99-a4d68bff5308 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:23.642736 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4b6d5-ed45-4a9d-808d-9e5e7b2607cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:23.643536 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4b6d5-ed45-4a9d-808d-9e5e7b2607cf None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:23.643975 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4b6d5-ed45-4a9d-808d-9e5e7b2607cf None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:23.644253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4b6d5-ed45-4a9d-808d-9e5e7b2607cf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:23.652464 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4b6d5-ed45-4a9d-808d-9e5e7b2607cf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:23.655130 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4b6d5-ed45-4a9d-808d-9e5e7b2607cf None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:23.671275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-a7e4b6d5-ed45-4a9d-808d-9e5e7b2607cf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:26:23.671275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:23.671275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:23.671275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:23.671275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:23.671275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:23.671275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:23.671275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:23.671275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:23.671275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:23.671275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:23.671275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:23.671275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:23.671275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:23.671275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:23.671275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:23.672254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:23.672254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:23.672254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:23.672254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:23.672254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:23.672254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:23.672254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:23.672254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:23.672254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:23.672254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:23.672254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:23.672254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:23.672254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:23.672254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:23.672254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:23.672254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:23.672254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:23.672254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:23.673685 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:23.673685 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:23.673685 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:23.673685 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:23.673685 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:23.673685 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:23.673685 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:23.673685 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:23.673685 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:26:23.673685 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:23.673685 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 383/787] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:26:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:23.678037 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6d97c-ecc7-46f1-86a5-f1601753ae6b None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:23.830929 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6d97c-ecc7-46f1-86a5-f1601753ae6b None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:23.845715 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:23.866222 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925cf89-04b8-4b9c-935b-94ca3440eed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:23.866931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925cf89-04b8-4b9c-935b-94ca3440eed3 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:23.867233 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925cf89-04b8-4b9c-935b-94ca3440eed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:23.867551 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925cf89-04b8-4b9c-935b-94ca3440eed3 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:23.868398 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a925cf89-04b8-4b9c-935b-94ca3440eed3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:23.868714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a925cf89-04b8-4b9c-935b-94ca3440eed3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:23.870147 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a925cf89-04b8-4b9c-935b-94ca3440eed3 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:23.889193 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 405/788] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Jun 23 11:26:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:23.894316 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4b6d5-ed45-4a9d-808d-9e5e7b2607cf None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:24.004970 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4b6d5-ed45-4a9d-808d-9e5e7b2607cf None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:24.039288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb450b-49ad-496f-9119-167a98e3c64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:24.040089 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb450b-49ad-496f-9119-167a98e3c64a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:24.040370 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb450b-49ad-496f-9119-167a98e3c64a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:24.040641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb450b-49ad-496f-9119-167a98e3c64a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:24.052434 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb450b-49ad-496f-9119-167a98e3c64a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:24.053882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb450b-49ad-496f-9119-167a98e3c64a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:24.062742 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:24.072460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-a3bb450b-49ad-496f-9119-167a98e3c64a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:24.072460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:24.072460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:24.072460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:24.072460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:24.072460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:24.072460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:24.072460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:24.072460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:24.072460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:24.072460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:24.072460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:24.072460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:24.072460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:24.072460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:24.073341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:24.073341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:24.073341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:24.073341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:24.073341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:24.073341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:24.073341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:24.073341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:24.073341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:24.073341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:24.073341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:24.073341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:24.073341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:24.073341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:24.073341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:24.073341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:24.073341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:24.074492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:24.074492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:24.074492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:24.074492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:24.074492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 384/789] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:24.079737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925cf89-04b8-4b9c-935b-94ca3440eed3 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:24.378225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925cf89-04b8-4b9c-935b-94ca3440eed3 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:24.439297 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82bcdb-60b9-460e-9b2c-d717761a1203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:24.439974 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82bcdb-60b9-460e-9b2c-d717761a1203 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:24.440256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82bcdb-60b9-460e-9b2c-d717761a1203 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:24.440529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82bcdb-60b9-460e-9b2c-d717761a1203 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:24.441257 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c82bcdb-60b9-460e-9b2c-d717761a1203 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:24.441563 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c82bcdb-60b9-460e-9b2c-d717761a1203 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:24.443213 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c82bcdb-60b9-460e-9b2c-d717761a1203 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:24.466646 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 406/790] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:24.475616 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb450b-49ad-496f-9119-167a98e3c64a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:24.596222 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288e296-b96b-4616-8457-c15454f69db0 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:24.596525 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288e296-b96b-4616-8457-c15454f69db0 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:24.596773 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288e296-b96b-4616-8457-c15454f69db0 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:24.598474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 407/791] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:24.608684 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527f3d3-a358-4585-8920-0eac5936eaef None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:24.608998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527f3d3-a358-4585-8920-0eac5936eaef None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:24.609250 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527f3d3-a358-4585-8920-0eac5936eaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:24.631414 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb450b-49ad-496f-9119-167a98e3c64a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:24.650462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-a527f3d3-a358-4585-8920-0eac5936eaef None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:24.651685 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a527f3d3-a358-4585-8920-0eac5936eaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:24.669057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ff249-6097-42e8-b48a-d3ca57d6a211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:24.670025 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ff249-6097-42e8-b48a-d3ca57d6a211 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:24.670308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ff249-6097-42e8-b48a-d3ca57d6a211 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:24.670618 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ff249-6097-42e8-b48a-d3ca57d6a211 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:24.688076 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ff249-6097-42e8-b48a-d3ca57d6a211 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:24.693940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ff249-6097-42e8-b48a-d3ca57d6a211 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:24.712003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:24.736429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-355ff249-6097-42e8-b48a-d3ca57d6a211 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:24.736429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:24.736429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:24.736429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:24.736429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:24.736429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:24.736429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:24.736429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:24.736429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:24.736429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:24.736429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:24.736429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:24.736429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:24.736429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:24.736429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:24.737846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:24.737846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:24.737846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:24.737846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:24.737846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:24.737846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:24.737846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:24.737846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:24.737846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:24.737846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:24.737846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:24.737846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:24.737846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:24.737846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:24.737846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:24.737846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:24.737846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:24.738771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:24.738771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:24.738771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:24.738771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:24.738771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:24.738771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:24.738771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:24.738771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:24.738771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 385/792] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:24.742259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa29c2a-c2a0-4687-891f-c0c4b669f888 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:24.742259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa29c2a-c2a0-4687-891f-c0c4b669f888 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:24.742259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa29c2a-c2a0-4687-891f-c0c4b669f888 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:24.747021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 386/793] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:24.748044 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa29c2a-c2a0-4687-891f-c0c4b669f888 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:24.767854 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 408/794] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:24.769225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0fe38-d874-4dbb-9494-8630cc6f8977 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:24.769816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0fe38-d874-4dbb-9494-8630cc6f8977 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:24.770481 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0fe38-d874-4dbb-9494-8630cc6f8977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:24.792231 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-eee0fe38-d874-4dbb-9494-8630cc6f8977 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:24.794421 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-eee0fe38-d874-4dbb-9494-8630cc6f8977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:24.822546 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:24.879944 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa29c2a-c2a0-4687-891f-c0c4b669f888 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:24.886733 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 409/795] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:24.887691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0fe38-d874-4dbb-9494-8630cc6f8977 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:24.925752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad9f77-4c63-4c64-ac98-ede88a6a0ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:24.927073 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad9f77-4c63-4c64-ac98-ede88a6a0ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:24.927533 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad9f77-4c63-4c64-ac98-ede88a6a0ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:24.927959 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad9f77-4c63-4c64-ac98-ede88a6a0ae8 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:24.928888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad9f77-4c63-4c64-ac98-ede88a6a0ae8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:24.929347 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad9f77-4c63-4c64-ac98-ede88a6a0ae8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:24.931374 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad9f77-4c63-4c64-ac98-ede88a6a0ae8 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:24.949239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 387/796] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:24.949239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56741e0c-93b4-4893-94c2-1f7a7e9d5096 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:24.949239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56741e0c-93b4-4893-94c2-1f7a7e9d5096 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:24.949239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56741e0c-93b4-4893-94c2-1f7a7e9d5096 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:24.949239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 388/797] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:24.949239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56741e0c-93b4-4893-94c2-1f7a7e9d5096 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:24.989234 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:24.997964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0fe38-d874-4dbb-9494-8630cc6f8977 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:25.035437 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61bff9-a858-4c43-9b66-cd18f77a93fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:25.036526 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61bff9-a858-4c43-9b66-cd18f77a93fa None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:25.036988 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61bff9-a858-4c43-9b66-cd18f77a93fa None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:25.037464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61bff9-a858-4c43-9b66-cd18f77a93fa None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:25.038296 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61bff9-a858-4c43-9b66-cd18f77a93fa None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:25.039775 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61bff9-a858-4c43-9b66-cd18f77a93fa None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:25.053024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-3b61bff9-a858-4c43-9b66-cd18f77a93fa None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 11:26:25.053024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:25.053024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:25.053024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:25.053024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:25.053024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:25.053024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:25.053024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:25.053024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 11:26:25.053024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:25.053024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:25.053024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:25.053024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:25.053024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:25.053024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:25.053024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:25.053984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:25.053984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:25.053984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:25.053984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:25.053984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:25.053984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:25.053984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:25.053984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:25.053984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:25.053984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:25.053984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:25.053984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:25.053984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:25.053984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:25.053984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:25.053984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:25.054866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:25.054866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:25.054866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:25.054866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:25.054866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:25.054866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:25.054866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:25.054866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:25.054866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:25.054866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:25.054866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:25.054866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:25.054866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:25.054866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 11:26:25.054866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:25.054866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 410/798] 10.222.0.13 () {56 vars in 1129 bytes} [Thu Jun 23 11:26:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:25.055724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61bff9-a858-4c43-9b66-cd18f77a93fa None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:25.073087 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56741e0c-93b4-4893-94c2-1f7a7e9d5096 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:25.094794 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:25.113242 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6674a-4fd3-4000-8afc-e1c4e450436a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:25.114042 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6674a-4fd3-4000-8afc-e1c4e450436a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:25.114325 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6674a-4fd3-4000-8afc-e1c4e450436a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:25.114627 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6674a-4fd3-4000-8afc-e1c4e450436a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:25.125862 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6674a-4fd3-4000-8afc-e1c4e450436a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:25.133402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-12b6674a-4fd3-4000-8afc-e1c4e450436a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:26:25.133402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:25.133402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:25.133402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:25.133402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:25.133402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:25.133402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:25.133402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:25.133402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:25.133402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:25.133402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:25.133402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:25.133402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:25.133402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:25.133402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:25.133402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:25.134445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:25.134445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:25.134445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:25.134445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:25.134445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:25.134445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:25.134445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:25.134445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:25.134445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:25.134445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:25.134445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:25.134445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:25.134445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:25.134445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:25.134445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:25.134445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:25.135322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:25.135322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:25.135322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:25.135322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:25.135322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:25.135322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:25.135322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:25.135322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:25.135322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:25.135322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:25.135322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:26:25.135322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:25.140374 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 389/799] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:26:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:25.141896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76118c-1f11-4ba1-b951-10a6ed38ad0a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:25.142381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76118c-1f11-4ba1-b951-10a6ed38ad0a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:25.142950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76118c-1f11-4ba1-b951-10a6ed38ad0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:25.161217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ad76118c-1f11-4ba1-b951-10a6ed38ad0a None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:25.162645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ad76118c-1f11-4ba1-b951-10a6ed38ad0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:25.170952 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61bff9-a858-4c43-9b66-cd18f77a93fa None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:25.210124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fc406-d262-49af-be27-af589f6fe9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:25.211039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fc406-d262-49af-be27-af589f6fe9b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:25.211317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fc406-d262-49af-be27-af589f6fe9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:25.211583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fc406-d262-49af-be27-af589f6fe9b6 None admin] PATH_INFO: `/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/450b1fad96604fb09d0705895c6d3da4/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:25.213714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1fc406-d262-49af-be27-af589f6fe9b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=886c2939a14b48febc2e2fe639c8f1e4, user_id=450b1fad96604fb09d0705895c6d3da4, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:25.215293 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1fc406-d262-49af-be27-af589f6fe9b6 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:25.234039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 390/800] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:25.234965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76118c-1f11-4ba1-b951-10a6ed38ad0a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:25.267485 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 411/801] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:26:25 2022] PUT /identity/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/450b1fad96604fb09d0705895c6d3da4/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:26:25.268373 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fc406-d262-49af-be27-af589f6fe9b6 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:25.292332 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:25.360551 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76118c-1f11-4ba1-b951-10a6ed38ad0a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:25.373761 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:25.399251 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf44e1-d84f-4028-93ad-9c32f666dcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:25.400242 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf44e1-d84f-4028-93ad-9c32f666dcb0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:25.400700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf44e1-d84f-4028-93ad-9c32f666dcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:25.401141 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf44e1-d84f-4028-93ad-9c32f666dcb0 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:25.402003 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf44e1-d84f-4028-93ad-9c32f666dcb0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:25.403132 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf44e1-d84f-4028-93ad-9c32f666dcb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:25.404850 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf44e1-d84f-4028-93ad-9c32f666dcb0 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:25.408009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fc406-d262-49af-be27-af589f6fe9b6 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:25.410751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 391/802] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Jun 23 11:26:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:25.411761 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf44e1-d84f-4028-93ad-9c32f666dcb0 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:25.462098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e5dd9-9368-455a-a975-e7fad6672955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:25.464772 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e5dd9-9368-455a-a975-e7fad6672955 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:25.465335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e5dd9-9368-455a-a975-e7fad6672955 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:25.466071 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e5dd9-9368-455a-a975-e7fad6672955 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:25.467493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e5dd9-9368-455a-a975-e7fad6672955 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:25.468249 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e5dd9-9368-455a-a975-e7fad6672955 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:25.470026 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e5dd9-9368-455a-a975-e7fad6672955 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:25.483660 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 412/803] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:26:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:25.484875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e5dd9-9368-455a-a975-e7fad6672955 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:25.534488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf44e1-d84f-4028-93ad-9c32f666dcb0 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:25.587489 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab82fb-4bac-4690-ad4c-b3f877131458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:25.588743 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:25.588948 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab82fb-4bac-4690-ad4c-b3f877131458 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:25.589642 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab82fb-4bac-4690-ad4c-b3f877131458 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:25.590569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab82fb-4bac-4690-ad4c-b3f877131458 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:25.602381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab82fb-4bac-4690-ad4c-b3f877131458 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:25.610909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-3cab82fb-4bac-4690-ad4c-b3f877131458 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:25.610909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:25.610909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:25.610909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:25.610909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:25.610909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:25.610909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:25.610909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:25.610909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:25.610909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:25.610909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:25.610909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:25.610909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:25.610909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:25.610909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:25.610909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:25.611843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:25.611843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:25.611843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:25.611843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:25.611843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:25.611843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:25.611843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:25.611843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:25.611843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:25.611843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:25.611843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:25.611843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:25.611843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:25.611843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:25.611843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:25.611843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:25.612670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:25.612670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:25.612670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:25.612670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:25.612670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:25.612670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:25.612670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:25.612670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:25.612670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:25.612670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:25.612670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:25.612670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:25.612670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 392/804] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:26:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:25.613365 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab82fb-4bac-4690-ad4c-b3f877131458 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:25.615573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e5dd9-9368-455a-a975-e7fad6672955 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:25.648835 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:25.652100 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eaae6a-8901-4167-9e1f-e749751d2a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:25.653241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaae6a-8901-4167-9e1f-e749751d2a2b None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:25.653528 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaae6a-8901-4167-9e1f-e749751d2a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:25.653813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaae6a-8901-4167-9e1f-e749751d2a2b None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:25.654670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eaae6a-8901-4167-9e1f-e749751d2a2b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:25.654991 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eaae6a-8901-4167-9e1f-e749751d2a2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:25.666272 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eaae6a-8901-4167-9e1f-e749751d2a2b None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:25.666272 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 413/805] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Jun 23 11:26:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:25.666272 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dd457-139c-4fb1-b371-a0d1ae783bf3 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:25.666272 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dd457-139c-4fb1-b371-a0d1ae783bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:25.666272 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dd457-139c-4fb1-b371-a0d1ae783bf3 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:25.666272 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 414/806] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:25.666272 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806dd457-139c-4fb1-b371-a0d1ae783bf3 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:25.734135 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab82fb-4bac-4690-ad4c-b3f877131458 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:25.770456 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250165ab-3955-4fc7-9786-c313db9e9be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:25.776720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250165ab-3955-4fc7-9786-c313db9e9be0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:25.776720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250165ab-3955-4fc7-9786-c313db9e9be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:25.776720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250165ab-3955-4fc7-9786-c313db9e9be0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:25.777040 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806dd457-139c-4fb1-b371-a0d1ae783bf3 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:25.784848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250165ab-3955-4fc7-9786-c313db9e9be0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:25.786323 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250165ab-3955-4fc7-9786-c313db9e9be0 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:25.805185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-250165ab-3955-4fc7-9786-c313db9e9be0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:25.805185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:25.805185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:25.805185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:25.805185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:25.805185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:25.805185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:25.805185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:25.805185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:25.805185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:25.805185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:25.805185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:25.805185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:25.805185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:25.805185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:25.806104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:25.806104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:25.806104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:25.806104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:25.806104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:25.806104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:25.806104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:25.806104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:25.806104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:25.806104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:25.806104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:25.806104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:25.806104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:25.806104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:25.806104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:25.806104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:25.806104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:25.807370 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:25.807370 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:25.807370 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:25.807370 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:25.807370 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 393/807] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:25.807370 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250165ab-3955-4fc7-9786-c313db9e9be0 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:25.817203 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e16e8-8541-4c24-aa43-55ddcafdf8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:25.818131 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e16e8-8541-4c24-aa43-55ddcafdf8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:25.818417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e16e8-8541-4c24-aa43-55ddcafdf8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:25.818952 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e16e8-8541-4c24-aa43-55ddcafdf8a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:25.819503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e16e8-8541-4c24-aa43-55ddcafdf8a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:25.819812 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e16e8-8541-4c24-aa43-55ddcafdf8a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:25.821175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e16e8-8541-4c24-aa43-55ddcafdf8a3 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:25.835406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 415/808] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:26:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:25.835406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867c444-8317-43c9-a6f6-13c769099f6a None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:25.835406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867c444-8317-43c9-a6f6-13c769099f6a None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:25.835406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867c444-8317-43c9-a6f6-13c769099f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:25.840216 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:25.853340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-8867c444-8317-43c9-a6f6-13c769099f6a None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:25.854839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-8867c444-8317-43c9-a6f6-13c769099f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:25.903049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:25.919641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250165ab-3955-4fc7-9786-c313db9e9be0 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:25.927788 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 416/809] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:25.928657 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867c444-8317-43c9-a6f6-13c769099f6a None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:25.967627 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ead54-b898-4c35-91bb-66823e01c057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:25.968877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ead54-b898-4c35-91bb-66823e01c057 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:25.969163 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ead54-b898-4c35-91bb-66823e01c057 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:25.969436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ead54-b898-4c35-91bb-66823e01c057 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:25.970188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ead54-b898-4c35-91bb-66823e01c057 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:25.971592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ead54-b898-4c35-91bb-66823e01c057 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:26.040011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867c444-8317-43c9-a6f6-13c769099f6a None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:26.074813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709c86f-57ed-4269-b14f-4c86e1a2b059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:26.075495 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709c86f-57ed-4269-b14f-4c86e1a2b059 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:26.075780 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709c86f-57ed-4269-b14f-4c86e1a2b059 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:26.076057 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709c86f-57ed-4269-b14f-4c86e1a2b059 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:26.076783 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709c86f-57ed-4269-b14f-4c86e1a2b059 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:26.077091 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709c86f-57ed-4269-b14f-4c86e1a2b059 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:26.078465 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709c86f-57ed-4269-b14f-4c86e1a2b059 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:26.079950 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:26.082650 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 394/810] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jun 23 11:26:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:26:26.083702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ead54-b898-4c35-91bb-66823e01c057 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:26.106718 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 417/811] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:26.107798 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709c86f-57ed-4269-b14f-4c86e1a2b059 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:26.220502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ead54-b898-4c35-91bb-66823e01c057 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:26.243604 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709c86f-57ed-4269-b14f-4c86e1a2b059 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:26.262955 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3ca73-fdbd-4549-9d9a-01304c89834f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:26.264054 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3ca73-fdbd-4549-9d9a-01304c89834f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:26.264493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3ca73-fdbd-4549-9d9a-01304c89834f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:26.264973 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3ca73-fdbd-4549-9d9a-01304c89834f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:26.272110 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:26.276790 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca3ca73-fdbd-4549-9d9a-01304c89834f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:26.278845 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca3ca73-fdbd-4549-9d9a-01304c89834f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:26.283712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e89a2-be7a-471f-a83e-2cd93c5aec0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:26.290510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e89a2-be7a-471f-a83e-2cd93c5aec0c None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:26.290510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e89a2-be7a-471f-a83e-2cd93c5aec0c None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:26.290510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e89a2-be7a-471f-a83e-2cd93c5aec0c None admin] PATH_INFO: `/v3/services/network` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:26.290510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e89a2-be7a-471f-a83e-2cd93c5aec0c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:26.290510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e89a2-be7a-471f-a83e-2cd93c5aec0c None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:26.297176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-eca3ca73-fdbd-4549-9d9a-01304c89834f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:26.297176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:26.297176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:26.297176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:26.297176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:26.297176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:26.297176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:26.297176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:26.297176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:26.297176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:26.297176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:26.297176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:26.297176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:26.297176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:26.297176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:26.298051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:26.298051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:26.298051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:26.298051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:26.298051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:26.298051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:26.298051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:26.298051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:26.298051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:26.298051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:26.298051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:26.298051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:26.298051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:26.298051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:26.298051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:26.298051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:26.298051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:26.299075 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:26.299075 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:26.299075 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:26.299075 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:26.299075 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 395/812] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:26.299764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3ca73-fdbd-4549-9d9a-01304c89834f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:26.303300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-c36e89a2-be7a-471f-a83e-2cd93c5aec0c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 11:26:26.303300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:26.303300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:26.303300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:26.303300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:26.303300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:26.303300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:26.303300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:26.303300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 11:26:26.303300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:26.303300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:26.303300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:26.303300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:26.303300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:26.303300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:26.303300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:26.303300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:26.304779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:26.304779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:26.304779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:26.304779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:26.304779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:26.304779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:26.304779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:26.304779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:26.304779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:26.304779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:26.304779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:26.304779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:26.304779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:26.304779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:26.304779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:26.304779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:26.305650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:26.305650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:26.305650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:26.305650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:26.305650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:26.305650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:26.305650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:26.305650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:26.305650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:26.305650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:26.305650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:26.305650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:26.305650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 11:26:26.305650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:26.305650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 418/813] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:26 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:26.308212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e89a2-be7a-471f-a83e-2cd93c5aec0c None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:26.384503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:26.422556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3ca73-fdbd-4549-9d9a-01304c89834f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:26.435942 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e89a2-be7a-471f-a83e-2cd93c5aec0c None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:26.463028 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8130eb-9a8d-4c2f-b6fa-352d3dfb6860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:26.463843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8130eb-9a8d-4c2f-b6fa-352d3dfb6860 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:26.464129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8130eb-9a8d-4c2f-b6fa-352d3dfb6860 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:26.464402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8130eb-9a8d-4c2f-b6fa-352d3dfb6860 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:26.473362 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8130eb-9a8d-4c2f-b6fa-352d3dfb6860 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:26.474762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8130eb-9a8d-4c2f-b6fa-352d3dfb6860 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:26.479213 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485434e4-2354-4c3a-bb25-54a39aa2ce06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:26.479886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485434e4-2354-4c3a-bb25-54a39aa2ce06 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:26.480175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485434e4-2354-4c3a-bb25-54a39aa2ce06 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:26.480454 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485434e4-2354-4c3a-bb25-54a39aa2ce06 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:26.481742 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485434e4-2354-4c3a-bb25-54a39aa2ce06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:26.482056 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485434e4-2354-4c3a-bb25-54a39aa2ce06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:26.489945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-4c8130eb-9a8d-4c2f-b6fa-352d3dfb6860 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:26:26.489945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:26.489945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:26.489945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:26.489945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:26.489945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:26.489945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:26.489945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:26.489945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:26.489945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:26.489945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:26.489945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:26.489945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:26.489945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:26.489945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:26.489945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:26.490929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:26.490929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:26.490929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:26.490929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:26.490929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:26.490929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:26.490929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:26.490929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:26.490929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:26.490929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:26.490929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:26.490929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:26.490929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:26.490929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:26.490929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:26.490929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:26.490929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:26.490929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:26.491861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:26.491861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:26.491861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:26.491861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:26.491861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:26.491861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:26.491861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:26.491861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:26.491861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:26:26.491861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:26.492385 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485434e4-2354-4c3a-bb25-54a39aa2ce06 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:26.497220 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 396/814] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jun 23 11:26:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:26.497220 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8130eb-9a8d-4c2f-b6fa-352d3dfb6860 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:26.506237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 419/815] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:26.507161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485434e4-2354-4c3a-bb25-54a39aa2ce06 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:26.551372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:26.602706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8130eb-9a8d-4c2f-b6fa-352d3dfb6860 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:26.619546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485434e4-2354-4c3a-bb25-54a39aa2ce06 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:26.637664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f029bd-cc4f-41a8-a0b0-fdbcddd04f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:26.638422 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f029bd-cc4f-41a8-a0b0-fdbcddd04f7e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:26.638816 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f029bd-cc4f-41a8-a0b0-fdbcddd04f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:26.639119 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:26.639286 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f029bd-cc4f-41a8-a0b0-fdbcddd04f7e None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:26.640346 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f029bd-cc4f-41a8-a0b0-fdbcddd04f7e None admin] RBAC: Adding query filter params (name=network) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:26.640742 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f029bd-cc4f-41a8-a0b0-fdbcddd04f7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:26.642188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f029bd-cc4f-41a8-a0b0-fdbcddd04f7e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:26.648753 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 397/816] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 23 11:26:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:26.654579 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f029bd-cc4f-41a8-a0b0-fdbcddd04f7e None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:26.659551 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146c3b5-b75a-41cb-a004-04db41e444b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:26.660682 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146c3b5-b75a-41cb-a004-04db41e444b6 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:26.661120 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146c3b5-b75a-41cb-a004-04db41e444b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:26.661553 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146c3b5-b75a-41cb-a004-04db41e444b6 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:26.662453 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146c3b5-b75a-41cb-a004-04db41e444b6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:26.662802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146c3b5-b75a-41cb-a004-04db41e444b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:26.664250 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146c3b5-b75a-41cb-a004-04db41e444b6 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:26.684114 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 420/817] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jun 23 11:26:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:26.684971 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146c3b5-b75a-41cb-a004-04db41e444b6 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:26.761273 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f029bd-cc4f-41a8-a0b0-fdbcddd04f7e None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:26.775888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:26.792357 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146c3b5-b75a-41cb-a004-04db41e444b6 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:26.797629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30057b-b7bc-420d-8129-08964142199f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:26.798705 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30057b-b7bc-420d-8129-08964142199f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:26.799170 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30057b-b7bc-420d-8129-08964142199f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:26.799587 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30057b-b7bc-420d-8129-08964142199f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:26.807585 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30057b-b7bc-420d-8129-08964142199f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:26.809249 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30057b-b7bc-420d-8129-08964142199f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:26.825179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-4c30057b-b7bc-420d-8129-08964142199f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:26:26.825179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:26.825179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:26.825179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:26.825179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:26.825179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:26.825179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:26.825179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:26.825179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:26.825179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:26.825179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:26.825179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:26.825179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:26.825179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:26.825179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:26.825179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:26.826493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:26.826493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:26.826493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:26.826493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:26.826493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:26.826493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:26.826493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:26.826493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:26.826493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:26.826493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:26.826493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:26.826493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:26.826493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:26.826493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:26.826493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:26.826493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:26.826493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:26.826493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:26.827677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:26.827677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:26.827677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:26.827677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:26.827677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:26.827677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:26.827677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:26.827677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:26.827677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:26:26.827677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:26.829628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b93af8-49ff-4685-8a28-588df8d84160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:26.830486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 398/818] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jun 23 11:26:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:26.834301 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b93af8-49ff-4685-8a28-588df8d84160 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:26.834301 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b93af8-49ff-4685-8a28-588df8d84160 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:26.834301 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b93af8-49ff-4685-8a28-588df8d84160 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:26.834805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30057b-b7bc-420d-8129-08964142199f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:26.834905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b93af8-49ff-4685-8a28-588df8d84160 None admin] RBAC: Adding query filter params (type=network) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:26.834905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b93af8-49ff-4685-8a28-588df8d84160 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:26.834905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b93af8-49ff-4685-8a28-588df8d84160 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:26.841200 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 421/819] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 23 11:26:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:26.842180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b93af8-49ff-4685-8a28-588df8d84160 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:26.889440 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:26.940156 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30057b-b7bc-420d-8129-08964142199f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:26.955416 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b93af8-49ff-4685-8a28-588df8d84160 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:26.976384 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b042a-f642-48fd-a0b8-88b4c3b0e0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:26.977642 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b042a-f642-48fd-a0b8-88b4c3b0e0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:26.978072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b042a-f642-48fd-a0b8-88b4c3b0e0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:26.978489 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b042a-f642-48fd-a0b8-88b4c3b0e0c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:26.994080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc3866-3b92-4a84-a064-e067b58628ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:26.995037 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3866-3b92-4a84-a064-e067b58628ee None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:26.995425 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3866-3b92-4a84-a064-e067b58628ee None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:26.995807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3866-3b92-4a84-a064-e067b58628ee None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:26.997006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc3866-3b92-4a84-a064-e067b58628ee None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:26.997419 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc3866-3b92-4a84-a064-e067b58628ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:26.999189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc3866-3b92-4a84-a064-e067b58628ee None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:27.019434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 422/820] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Jun 23 11:26:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:27.020326 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc3866-3b92-4a84-a064-e067b58628ee None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:27.025119 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:27.051740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b042a-f642-48fd-a0b8-88b4c3b0e0c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:27.053233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b042a-f642-48fd-a0b8-88b4c3b0e0c2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:27.128243 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:27.133085 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc3866-3b92-4a84-a064-e067b58628ee None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:27.160749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 399/821] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:26:26 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:26:27.161698 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b042a-f642-48fd-a0b8-88b4c3b0e0c2 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:27.171095 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1dc70-9217-49e3-bf14-be24040c7c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:27.172097 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1dc70-9217-49e3-bf14-be24040c7c30 None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:27.172713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1dc70-9217-49e3-bf14-be24040c7c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:27.173199 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1dc70-9217-49e3-bf14-be24040c7c30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:27.175342 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1dc70-9217-49e3-bf14-be24040c7c30 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:27.177061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1dc70-9217-49e3-bf14-be24040c7c30 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:27.240392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:27.268571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b042a-f642-48fd-a0b8-88b4c3b0e0c2 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:27.274780 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 423/822] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jun 23 11:26:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:26:27.275730 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1dc70-9217-49e3-bf14-be24040c7c30 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:27.324875 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694dcb0-c47e-4146-a2ff-98471b8399b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:27.326153 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694dcb0-c47e-4146-a2ff-98471b8399b6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:27.326721 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694dcb0-c47e-4146-a2ff-98471b8399b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:27.327206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694dcb0-c47e-4146-a2ff-98471b8399b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:27.341516 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694dcb0-c47e-4146-a2ff-98471b8399b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:27.344317 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694dcb0-c47e-4146-a2ff-98471b8399b6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:27.382695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-a694dcb0-c47e-4146-a2ff-98471b8399b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:27.382695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:27.382695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:27.382695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:27.382695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:27.382695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:27.382695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:27.382695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:27.382695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:27.382695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:27.382695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:27.382695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:27.382695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:27.382695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:27.382695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:27.383679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:27.383679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:27.383679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:27.383679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:27.383679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:27.383679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:27.383679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:27.383679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:27.383679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:27.383679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:27.383679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:27.383679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:27.383679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:27.383679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:27.383679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:27.383679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:27.383679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:27.384606 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:27.384606 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:27.384606 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:27.384606 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:27.388354 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 400/823] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:27.394183 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694dcb0-c47e-4146-a2ff-98471b8399b6 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:27.501005 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1dc70-9217-49e3-bf14-be24040c7c30 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:27.542844 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694dcb0-c47e-4146-a2ff-98471b8399b6 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:27.553974 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f61a5-6f99-4eb2-abc8-3f653b2e71de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:27.555175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f61a5-6f99-4eb2-abc8-3f653b2e71de None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:27.555463 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f61a5-6f99-4eb2-abc8-3f653b2e71de None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:27.555732 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f61a5-6f99-4eb2-abc8-3f653b2e71de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:27.567735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f61a5-6f99-4eb2-abc8-3f653b2e71de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:27.568943 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:27.569189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f61a5-6f99-4eb2-abc8-3f653b2e71de None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:27.582037 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279fb223-a07b-4995-8e68-fb82855ad67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:27.582760 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fb223-a07b-4995-8e68-fb82855ad67e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:27.583051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fb223-a07b-4995-8e68-fb82855ad67e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:27.583325 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fb223-a07b-4995-8e68-fb82855ad67e None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:27.584049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279fb223-a07b-4995-8e68-fb82855ad67e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:27.584360 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279fb223-a07b-4995-8e68-fb82855ad67e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:27.585819 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279fb223-a07b-4995-8e68-fb82855ad67e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:27.588190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-1f0f61a5-6f99-4eb2-abc8-3f653b2e71de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:27.588190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:27.588190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:27.588190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:27.588190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:27.588190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:27.588190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:27.588190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:27.588190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:27.588190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:27.588190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:27.588190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:27.588190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:27.588190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:27.588190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:27.594800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:27.594800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:27.594800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:27.594800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:27.594800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:27.594800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:27.594800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:27.594800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:27.594800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:27.594800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:27.594800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:27.594800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:27.594800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:27.594800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:27.594800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:27.594800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:27.594800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:27.595713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:27.595713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:27.595713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:27.595713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:27.595713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 424/824] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:27.595713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc1d43-1ae5-4d45-ae39-5ce0bae3f6d8 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:27.595713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc1d43-1ae5-4d45-ae39-5ce0bae3f6d8 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:27.595713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc1d43-1ae5-4d45-ae39-5ce0bae3f6d8 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:27.595713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 425/825] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:27.597945 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc1d43-1ae5-4d45-ae39-5ce0bae3f6d8 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:27.601931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 401/826] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:27.605131 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e03e5-ca35-40ef-80c2-25c3bdb5afbb None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:27.605131 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e03e5-ca35-40ef-80c2-25c3bdb5afbb None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:27.605131 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e03e5-ca35-40ef-80c2-25c3bdb5afbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:27.608762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:27.620388 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-969e03e5-ca35-40ef-80c2-25c3bdb5afbb None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:27.621911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-969e03e5-ca35-40ef-80c2-25c3bdb5afbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:27.692181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 402/827] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:27.693490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e03e5-ca35-40ef-80c2-25c3bdb5afbb None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:27.723869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc1d43-1ae5-4d45-ae39-5ce0bae3f6d8 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:27.760907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1fdc6b-096d-452b-843f-1962e2998f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:27.762587 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fdc6b-096d-452b-843f-1962e2998f59 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:27.763080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fdc6b-096d-452b-843f-1962e2998f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:27.763526 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fdc6b-096d-452b-843f-1962e2998f59 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:27.766965 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1fdc6b-096d-452b-843f-1962e2998f59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:27.766965 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1fdc6b-096d-452b-843f-1962e2998f59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:27.766965 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1fdc6b-096d-452b-843f-1962e2998f59 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:27.779267 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 426/828] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:27.780364 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1fdc6b-096d-452b-843f-1962e2998f59 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:27.811563 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e03e5-ca35-40ef-80c2-25c3bdb5afbb None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:27.826200 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:27.848861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef74e58-4287-4cfd-9051-c10102c915cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:27.849729 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef74e58-4287-4cfd-9051-c10102c915cf None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:27.850078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef74e58-4287-4cfd-9051-c10102c915cf None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:27.850418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef74e58-4287-4cfd-9051-c10102c915cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:27.853495 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:27.857702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef74e58-4287-4cfd-9051-c10102c915cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:27.859189 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef74e58-4287-4cfd-9051-c10102c915cf None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:27.874488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-fef74e58-4287-4cfd-9051-c10102c915cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:27.874488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:27.874488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:27.874488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:27.874488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:27.874488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:27.874488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:27.874488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:27.874488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:27.874488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:27.874488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:27.874488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:27.874488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:27.874488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:27.874488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:27.875605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:27.875605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:27.875605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:27.875605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:27.875605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:27.875605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:27.875605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:27.875605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:27.875605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:27.875605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:27.875605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:27.875605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:27.875605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:27.875605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:27.875605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:27.875605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:27.875605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:27.876950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:27.876950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:27.876950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:27.876950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:27.876950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:27.876950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:27.876950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:27.876950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:27.880201 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 403/829] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:27.880974 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef74e58-4287-4cfd-9051-c10102c915cf None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:27.891610 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1fdc6b-096d-452b-843f-1962e2998f59 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:27.927274 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018961fc-587d-42af-955a-4d1d12b76d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:27.928076 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018961fc-587d-42af-955a-4d1d12b76d49 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:27.928362 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018961fc-587d-42af-955a-4d1d12b76d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:27.928641 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018961fc-587d-42af-955a-4d1d12b76d49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:27.939489 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018961fc-587d-42af-955a-4d1d12b76d49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:27.947346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-018961fc-587d-42af-955a-4d1d12b76d49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:26:27.947346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:27.947346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:27.947346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:27.947346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:27.947346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:27.947346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:27.947346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:27.947346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:27.947346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:27.947346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:27.947346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:27.947346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:27.947346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:27.947346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:27.947346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:27.948289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:27.948289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:27.948289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:27.948289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:27.948289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:27.948289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:27.948289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:27.948289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:27.948289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:27.948289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:27.948289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:27.948289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:27.948289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:27.948289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:27.948289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:27.948289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:27.949136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:27.949136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:27.949136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:27.949136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:27.949136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:27.949136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:27.949136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:27.949136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:27.949136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:27.949136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:27.949136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:26:27.949136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:27.949136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 427/830] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:26:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:27.950432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018961fc-587d-42af-955a-4d1d12b76d49 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:27.994983 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef74e58-4287-4cfd-9051-c10102c915cf None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:28.036468 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4618e-e0c6-4a6b-9518-9e5c7be2e3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:28.037319 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4618e-e0c6-4a6b-9518-9e5c7be2e3ae None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:28.037603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4618e-e0c6-4a6b-9518-9e5c7be2e3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:28.037896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4618e-e0c6-4a6b-9518-9e5c7be2e3ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:28.045593 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd4618e-e0c6-4a6b-9518-9e5c7be2e3ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:28.047076 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd4618e-e0c6-4a6b-9518-9e5c7be2e3ae None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:28.061752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-5cd4618e-e0c6-4a6b-9518-9e5c7be2e3ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:28.061752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:28.061752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:28.061752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:28.061752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:28.061752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:28.061752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:28.061752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:28.061752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:28.061752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:28.061752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:28.061752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:28.061752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:28.061752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:28.061752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:28.062684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:28.062684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:28.062684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:28.062684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:28.062684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:28.062684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:28.062684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:28.062684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:28.062684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:28.062684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:28.062684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:28.062684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:28.062684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:28.062684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:28.062684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:28.062684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:28.062684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:28.063599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:28.063599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:28.063599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:28.063599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:28.063599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:28.063599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:28.063599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:28.063599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:28.063599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 404/831] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:28.064083 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4618e-e0c6-4a6b-9518-9e5c7be2e3ae None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:28.071476 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018961fc-587d-42af-955a-4d1d12b76d49 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:28.098612 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:28.100689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:28.110011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc039bde-300d-4f3a-8bee-d961fcdfe38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:28.110749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc039bde-300d-4f3a-8bee-d961fcdfe38e None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:28.111008 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc039bde-300d-4f3a-8bee-d961fcdfe38e None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:28.111645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc039bde-300d-4f3a-8bee-d961fcdfe38e None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:28.112195 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc039bde-300d-4f3a-8bee-d961fcdfe38e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:28.112655 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc039bde-300d-4f3a-8bee-d961fcdfe38e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:28.114098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc039bde-300d-4f3a-8bee-d961fcdfe38e None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:28.130964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 428/832] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:28.130964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc039bde-300d-4f3a-8bee-d961fcdfe38e None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:28.183434 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4618e-e0c6-4a6b-9518-9e5c7be2e3ae None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:28.222872 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e802166-b242-4582-9dfc-92f84d3f26c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:28.223543 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e802166-b242-4582-9dfc-92f84d3f26c2 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:28.223831 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e802166-b242-4582-9dfc-92f84d3f26c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:28.224109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e802166-b242-4582-9dfc-92f84d3f26c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:28.224861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e802166-b242-4582-9dfc-92f84d3f26c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:28.225178 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e802166-b242-4582-9dfc-92f84d3f26c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:28.226568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e802166-b242-4582-9dfc-92f84d3f26c2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:28.231114 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc039bde-300d-4f3a-8bee-d961fcdfe38e None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:28.240294 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 405/833] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:26:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:28.240294 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e802166-b242-4582-9dfc-92f84d3f26c2 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:28.268796 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0385a-0813-435d-b080-a1980f49f6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:28.269545 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0385a-0813-435d-b080-a1980f49f6fe None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:28.269899 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0385a-0813-435d-b080-a1980f49f6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:28.270250 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0385a-0813-435d-b080-a1980f49f6fe None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:28.271073 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0385a-0813-435d-b080-a1980f49f6fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:28.271382 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0385a-0813-435d-b080-a1980f49f6fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:28.272818 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0385a-0813-435d-b080-a1980f49f6fe None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:28.290035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 429/834] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:28.290035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0385a-0813-435d-b080-a1980f49f6fe None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:28.336568 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:28.345792 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:28.352487 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e802166-b242-4582-9dfc-92f84d3f26c2 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:28.385510 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7210adc-a572-494c-b4a6-4d61334d750e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:28.386396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7210adc-a572-494c-b4a6-4d61334d750e None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:28.387319 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7210adc-a572-494c-b4a6-4d61334d750e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:28.387625 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7210adc-a572-494c-b4a6-4d61334d750e None admin] PATH_INFO: `/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/030614ef56f143f8be6fcc0445ec9568/roles/13708a4819a94b94b47ba5994aaa9310` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:28.391536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0385a-0813-435d-b080-a1980f49f6fe None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:28.401929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7210adc-a572-494c-b4a6-4d61334d750e None admin] RBAC: Authorizing `identity:create_grant(project_id=886c2939a14b48febc2e2fe639c8f1e4, user_id=030614ef56f143f8be6fcc0445ec9568, role_id=13708a4819a94b94b47ba5994aaa9310)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:28.404038 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7210adc-a572-494c-b4a6-4d61334d750e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:28.421568 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec5d42-78de-49c7-b078-625fe86246df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:28.422352 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec5d42-78de-49c7-b078-625fe86246df None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:28.422670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec5d42-78de-49c7-b078-625fe86246df None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:28.422948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec5d42-78de-49c7-b078-625fe86246df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:28.433168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec5d42-78de-49c7-b078-625fe86246df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:28.434675 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec5d42-78de-49c7-b078-625fe86246df None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:28.452118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-23ec5d42-78de-49c7-b078-625fe86246df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:28.452118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:28.452118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:28.452118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:28.452118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:28.452118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:28.452118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:28.452118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:28.452118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:28.452118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:28.452118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:28.452118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:28.452118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:28.452118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:28.452118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:28.452981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:28.452981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:28.452981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:28.452981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:28.452981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:28.452981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:28.452981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:28.452981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:28.452981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:28.452981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:28.452981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:28.452981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:28.452981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:28.452981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:28.452981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:28.452981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:28.452981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:28.453865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:28.453865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:28.453865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:28.453865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:28.453865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 430/835] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:28.454136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec5d42-78de-49c7-b078-625fe86246df None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:28.472382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 406/836] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:26:28 2022] PUT /identity/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/030614ef56f143f8be6fcc0445ec9568/roles/13708a4819a94b94b47ba5994aaa9310 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:26:28.473737 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d0350-dae9-4427-b636-e98e1807f6ba None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:28.474036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d0350-dae9-4427-b636-e98e1807f6ba None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:28.474483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d0350-dae9-4427-b636-e98e1807f6ba None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:28.478739 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 407/837] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:28.478739 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d0350-dae9-4427-b636-e98e1807f6ba None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:28.575260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:28.580480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec5d42-78de-49c7-b078-625fe86246df None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:28.608340 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d0350-dae9-4427-b636-e98e1807f6ba None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:28.625456 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276582d0-06ac-4a0a-99ce-7c855ea70df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:28.625456 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276582d0-06ac-4a0a-99ce-7c855ea70df8 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:28.625456 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276582d0-06ac-4a0a-99ce-7c855ea70df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:28.625815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276582d0-06ac-4a0a-99ce-7c855ea70df8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:28.627701 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276582d0-06ac-4a0a-99ce-7c855ea70df8 None admin] RBAC: Adding query filter params (role.id=c9e006de0d6c4812ab29d778e9c8a830, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=161c5ed1e47e450290a1f428ee100b68) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:28.627701 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276582d0-06ac-4a0a-99ce-7c855ea70df8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:28.628365 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276582d0-06ac-4a0a-99ce-7c855ea70df8 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:28.636411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 431/838] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:26:28 2022] GET /identity/v3/role_assignments?user.id=161c5ed1e47e450290a1f428ee100b68&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=c9e006de0d6c4812ab29d778e9c8a830 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:28.638393 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4de6a-1cd0-431d-9a80-4d655f2eeeb1 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:28.638708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4de6a-1cd0-431d-9a80-4d655f2eeeb1 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:28.638988 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4de6a-1cd0-431d-9a80-4d655f2eeeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:28.656181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4839e-db98-4d0a-9386-fab941abdd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:28.656691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4839e-db98-4d0a-9386-fab941abdd3b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:28.656969 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4839e-db98-4d0a-9386-fab941abdd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:28.657235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4839e-db98-4d0a-9386-fab941abdd3b None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:28.657925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-ade4de6a-1cd0-431d-9a80-4d655f2eeeb1 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:28.658027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b4839e-db98-4d0a-9386-fab941abdd3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:28.658273 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b4839e-db98-4d0a-9386-fab941abdd3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:28.661401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-ade4de6a-1cd0-431d-9a80-4d655f2eeeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:28.661576 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b4839e-db98-4d0a-9386-fab941abdd3b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:28.680287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 408/839] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:28.686708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4839e-db98-4d0a-9386-fab941abdd3b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:28.690999 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:28.745597 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 432/840] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:28.768039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4de6a-1cd0-431d-9a80-4d655f2eeeb1 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:28.802231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4839e-db98-4d0a-9386-fab941abdd3b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:28.848216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a92dc5-4ab7-451d-8084-b1a22928f8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:28.849830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a92dc5-4ab7-451d-8084-b1a22928f8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:28.850123 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a92dc5-4ab7-451d-8084-b1a22928f8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:28.850404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a92dc5-4ab7-451d-8084-b1a22928f8e5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:28.863278 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a92dc5-4ab7-451d-8084-b1a22928f8e5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:28.865074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a92dc5-4ab7-451d-8084-b1a22928f8e5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:28.874352 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:28.881922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-d7a92dc5-4ab7-451d-8084-b1a22928f8e5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:26:28.881922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:28.881922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:28.881922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:28.881922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:28.881922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:28.881922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:28.881922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:28.881922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:28.881922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:28.881922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:28.881922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:28.881922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:28.881922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:28.881922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:28.881922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:28.883180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:28.883180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:28.883180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:28.883180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:28.883180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:28.883180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:28.883180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:28.883180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:28.883180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:28.883180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:28.883180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:28.883180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:28.883180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:28.883180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:28.883180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:28.883180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:28.883180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:28.883180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:28.885540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:28.885540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:28.885540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:28.885540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:28.885540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:28.885540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:28.885540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:28.885540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:28.885540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:26:28.885540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:28.885540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 409/841] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:26:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:28.892244 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a92dc5-4ab7-451d-8084-b1a22928f8e5 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:28.892894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4de6a-1cd0-431d-9a80-4d655f2eeeb1 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:28.929487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13464ac-b096-411e-a3b8-538d625a386a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:28.930285 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13464ac-b096-411e-a3b8-538d625a386a None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:28.930595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13464ac-b096-411e-a3b8-538d625a386a None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:28.930993 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13464ac-b096-411e-a3b8-538d625a386a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:28.931684 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13464ac-b096-411e-a3b8-538d625a386a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:28.933049 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13464ac-b096-411e-a3b8-538d625a386a None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:28.945880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-a13464ac-b096-411e-a3b8-538d625a386a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 11:26:28.945880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:28.945880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:28.945880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:28.945880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:28.945880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:28.945880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:28.945880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:28.945880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 11:26:28.945880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:28.945880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:28.945880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:28.945880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:28.945880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:28.945880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:28.945880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:28.945880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:28.947009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:28.947009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:28.947009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:28.947009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:28.947009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:28.947009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:28.947009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:28.947009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:28.947009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:28.947009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:28.947009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:28.947009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:28.947009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:28.947009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:28.947009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:28.947009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:28.947858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:28.947858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:28.947858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:28.947858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:28.947858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:28.947858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:28.947858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:28.947858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:28.947858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:28.947858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:28.947858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:28.947858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:28.947858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 11:26:28.947858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:28.947858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 433/842] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:26:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:28.955678 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13464ac-b096-411e-a3b8-538d625a386a None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:28.988129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:29.023855 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a92dc5-4ab7-451d-8084-b1a22928f8e5 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:29.071011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20637ba-9968-4b2b-bac9-4c8010c6228b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:29.072363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20637ba-9968-4b2b-bac9-4c8010c6228b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:29.072859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20637ba-9968-4b2b-bac9-4c8010c6228b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:29.073596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20637ba-9968-4b2b-bac9-4c8010c6228b None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:29.075456 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20637ba-9968-4b2b-bac9-4c8010c6228b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:29.075873 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20637ba-9968-4b2b-bac9-4c8010c6228b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:29.077337 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20637ba-9968-4b2b-bac9-4c8010c6228b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:29.087824 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13464ac-b096-411e-a3b8-538d625a386a None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:29.096723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 410/843] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Jun 23 11:26:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:29.101852 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20637ba-9968-4b2b-bac9-4c8010c6228b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:29.125536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0851140-1453-4588-93db-c59896bfc86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:29.126451 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0851140-1453-4588-93db-c59896bfc86f None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:29.126928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0851140-1453-4588-93db-c59896bfc86f None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:29.127355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0851140-1453-4588-93db-c59896bfc86f None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:29.128201 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0851140-1453-4588-93db-c59896bfc86f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:29.128657 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0851140-1453-4588-93db-c59896bfc86f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:29.130251 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0851140-1453-4588-93db-c59896bfc86f None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:29.135550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 434/844] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:26:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:29.140564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0851140-1453-4588-93db-c59896bfc86f None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:29.166943 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:29.216491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20637ba-9968-4b2b-bac9-4c8010c6228b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:29.240536 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:29.253384 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40866ba-ef58-421a-bcb3-8409fa5b0892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:29.254363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40866ba-ef58-421a-bcb3-8409fa5b0892 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:29.254767 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0851140-1453-4588-93db-c59896bfc86f None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:29.255083 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40866ba-ef58-421a-bcb3-8409fa5b0892 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:29.255556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40866ba-ef58-421a-bcb3-8409fa5b0892 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:29.293208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2e5f2-9de2-4b25-87a5-204b46814e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:29.293860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2e5f2-9de2-4b25-87a5-204b46814e39 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:29.294147 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2e5f2-9de2-4b25-87a5-204b46814e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:29.294432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2e5f2-9de2-4b25-87a5-204b46814e39 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:29.295395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2e5f2-9de2-4b25-87a5-204b46814e39 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:29.295652 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2e5f2-9de2-4b25-87a5-204b46814e39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:29.297250 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2e5f2-9de2-4b25-87a5-204b46814e39 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:29.307019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 435/845] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:26:29 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:29.340538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40866ba-ef58-421a-bcb3-8409fa5b0892 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:29.342724 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40866ba-ef58-421a-bcb3-8409fa5b0892 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:29.455208 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 411/846] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:26:29 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:26:29.461154 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2e5f2-9de2-4b25-87a5-204b46814e39 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:29.553434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:29.569646 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2e5f2-9de2-4b25-87a5-204b46814e39 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:29.606880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e5b3a-c877-4936-bd5a-b0c067ff2690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:29.607679 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e5b3a-c877-4936-bd5a-b0c067ff2690 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:29.607969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e5b3a-c877-4936-bd5a-b0c067ff2690 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:29.608244 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e5b3a-c877-4936-bd5a-b0c067ff2690 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:29.618826 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e5b3a-c877-4936-bd5a-b0c067ff2690 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:29.620283 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e5b3a-c877-4936-bd5a-b0c067ff2690 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:29.638292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-398e5b3a-c877-4936-bd5a-b0c067ff2690 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:29.638292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:29.638292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:29.638292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:29.638292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:29.638292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:29.638292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:29.638292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:29.638292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:29.638292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:29.638292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:29.638292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:29.638292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:29.638292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:29.638292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:29.639225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:29.639225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:29.639225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:29.639225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:29.639225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:29.639225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:29.639225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:29.639225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:29.639225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:29.639225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:29.639225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:29.639225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:29.639225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:29.639225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:29.639225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:29.639225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:29.639225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:29.640284 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:29.640284 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:29.640284 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:29.640284 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:29.640284 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 436/847] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:29.645273 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40866ba-ef58-421a-bcb3-8409fa5b0892 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:29.672155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:29.752095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40866ba-ef58-421a-bcb3-8409fa5b0892 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:29.787749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c659ac-3071-4435-9fb4-77100cefb5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:29.788474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c659ac-3071-4435-9fb4-77100cefb5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:29.788774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c659ac-3071-4435-9fb4-77100cefb5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:29.789051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c659ac-3071-4435-9fb4-77100cefb5f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:29.789765 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c659ac-3071-4435-9fb4-77100cefb5f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:29.790070 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c659ac-3071-4435-9fb4-77100cefb5f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:29.791468 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c659ac-3071-4435-9fb4-77100cefb5f3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:29.803541 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 412/848] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:29.808838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e5b3a-c877-4936-bd5a-b0c067ff2690 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:29.914794 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e5b3a-c877-4936-bd5a-b0c067ff2690 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:29.951684 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759e546-4cab-4e05-ae44-ce6156cb8fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:29.952837 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759e546-4cab-4e05-ae44-ce6156cb8fab None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:29.953334 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759e546-4cab-4e05-ae44-ce6156cb8fab None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:29.953838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759e546-4cab-4e05-ae44-ce6156cb8fab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:29.956842 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:29.961286 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759e546-4cab-4e05-ae44-ce6156cb8fab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:29.962771 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759e546-4cab-4e05-ae44-ce6156cb8fab None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:29.977795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-c759e546-4cab-4e05-ae44-ce6156cb8fab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:29.977795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:29.977795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:29.977795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:29.977795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:29.977795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:29.977795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:29.977795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:29.977795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:29.977795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:29.977795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:29.977795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:29.977795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:29.977795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:29.977795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:29.980806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:29.980806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:29.980806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:29.980806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:29.980806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:29.980806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:29.980806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:29.980806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:29.980806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:29.980806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:29.980806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:29.980806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:29.980806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:29.980806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:29.980806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:29.980806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:29.980806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:29.981712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:29.981712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:29.981712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:29.981712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:29.981712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:29.981712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:29.981712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:29.981712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:29.981712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 437/849] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:29.987543 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c659ac-3071-4435-9fb4-77100cefb5f3 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:30.075572 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:30.093923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c659ac-3071-4435-9fb4-77100cefb5f3 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:30.129006 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1629658-8c11-4f4c-b563-6a89f5347edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:30.130201 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1629658-8c11-4f4c-b563-6a89f5347edd None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:30.134903 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1629658-8c11-4f4c-b563-6a89f5347edd None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:30.135351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1629658-8c11-4f4c-b563-6a89f5347edd None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:30.136281 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1629658-8c11-4f4c-b563-6a89f5347edd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:30.136995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1629658-8c11-4f4c-b563-6a89f5347edd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:30.138709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1629658-8c11-4f4c-b563-6a89f5347edd None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:30.151161 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 413/850] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:30.162647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759e546-4cab-4e05-ae44-ce6156cb8fab None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:30.265237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759e546-4cab-4e05-ae44-ce6156cb8fab None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:30.300802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c00b8-6857-4894-9f34-cdff3d4a1b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:30.301488 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c00b8-6857-4894-9f34-cdff3d4a1b4a None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:30.301783 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c00b8-6857-4894-9f34-cdff3d4a1b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:30.302061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c00b8-6857-4894-9f34-cdff3d4a1b4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:30.302933 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c00b8-6857-4894-9f34-cdff3d4a1b4a None admin] RBAC: Adding query filter params (role.id=c9e006de0d6c4812ab29d778e9c8a830, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=450b1fad96604fb09d0705895c6d3da4) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:30.303249 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c00b8-6857-4894-9f34-cdff3d4a1b4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:30.304681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c00b8-6857-4894-9f34-cdff3d4a1b4a None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:30.311696 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 438/851] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:26:30 2022] GET /identity/v3/role_assignments?user.id=450b1fad96604fb09d0705895c6d3da4&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=c9e006de0d6c4812ab29d778e9c8a830 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:30.681399 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0487df-4cc8-4430-bff8-34d7acdc686a None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:30.681772 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0487df-4cc8-4430-bff8-34d7acdc686a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:30.681967 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0487df-4cc8-4430-bff8-34d7acdc686a None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:30.684630 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 414/852] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:30.689862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3be9b7-8a34-48f2-a6c4-9106e91033b8 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:30.690160 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3be9b7-8a34-48f2-a6c4-9106e91033b8 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:30.690438 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3be9b7-8a34-48f2-a6c4-9106e91033b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:30.708144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-be3be9b7-8a34-48f2-a6c4-9106e91033b8 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:30.709310 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-be3be9b7-8a34-48f2-a6c4-9106e91033b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:30.781383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:30.781383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 439/853] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:30.805281 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0487df-4cc8-4430-bff8-34d7acdc686a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:30.897577 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057127f4-db26-4a0c-a8b3-f7cb21a1e572 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:30.897904 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057127f4-db26-4a0c-a8b3-f7cb21a1e572 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:30.898134 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057127f4-db26-4a0c-a8b3-f7cb21a1e572 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:30.899849 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 440/854] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:30.905246 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc39a2-c86f-41c2-870f-e6961edcfa9e None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:30.905533 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc39a2-c86f-41c2-870f-e6961edcfa9e None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:30.905807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc39a2-c86f-41c2-870f-e6961edcfa9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:30.916940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0487df-4cc8-4430-bff8-34d7acdc686a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:30.923386 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-1ebc39a2-c86f-41c2-870f-e6961edcfa9e None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:30.925248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-1ebc39a2-c86f-41c2-870f-e6961edcfa9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:30.958022 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180be90-c01e-4e0f-b8af-757445936cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:30.959330 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180be90-c01e-4e0f-b8af-757445936cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:30.960026 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180be90-c01e-4e0f-b8af-757445936cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:30.960444 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180be90-c01e-4e0f-b8af-757445936cb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:30.974031 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180be90-c01e-4e0f-b8af-757445936cb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:30.985780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-2180be90-c01e-4e0f-b8af-757445936cb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:30.985780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:30.985780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:30.985780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:30.985780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:30.985780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:30.985780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:30.985780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:30.985780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:30.985780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:30.985780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:30.985780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:30.985780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:30.985780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:30.985780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:30.985780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:30.986931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:30.986931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:30.986931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:30.986931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:30.986931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:30.986931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:30.986931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:30.986931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:30.986931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:30.986931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:30.986931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:30.986931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:30.986931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:30.986931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:30.986931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:30.986931 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:30.987958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:30.987958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:30.987958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:30.987958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:30.987958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:30.987958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:30.987958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:30.987958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:30.987958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:30.987958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:30.987958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:26:30.987958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:30.995453 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 415/855] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:26:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:30.997081 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180be90-c01e-4e0f-b8af-757445936cb3 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:31.002645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:31.022567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 441/856] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:26:31.040505 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc39a2-c86f-41c2-870f-e6961edcfa9e None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:31.116138 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180be90-c01e-4e0f-b8af-757445936cb3 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:31.151160 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc39a2-c86f-41c2-870f-e6961edcfa9e None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:31.162346 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271df53-91d1-4a0e-9058-107b5890846b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:31.163302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271df53-91d1-4a0e-9058-107b5890846b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:31.164127 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271df53-91d1-4a0e-9058-107b5890846b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:31.164548 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271df53-91d1-4a0e-9058-107b5890846b None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:31.165885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271df53-91d1-4a0e-9058-107b5890846b None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:31.166627 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271df53-91d1-4a0e-9058-107b5890846b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:31.169095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271df53-91d1-4a0e-9058-107b5890846b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:31.177085 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:31.185450 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 416/857] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:26:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:31.192649 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271df53-91d1-4a0e-9058-107b5890846b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:31.195606 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e883a08-8234-48b6-be30-33c2a4de9f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:31.196596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e883a08-8234-48b6-be30-33c2a4de9f76 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:31.196983 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e883a08-8234-48b6-be30-33c2a4de9f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:31.197379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e883a08-8234-48b6-be30-33c2a4de9f76 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:31.198148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e883a08-8234-48b6-be30-33c2a4de9f76 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:31.199873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e883a08-8234-48b6-be30-33c2a4de9f76 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:31.213760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-0e883a08-8234-48b6-be30-33c2a4de9f76 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 11:26:31.213760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:31.213760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:31.213760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:31.213760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:31.213760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:31.213760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:31.213760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:31.213760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 11:26:31.213760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:31.213760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:31.213760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:31.213760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:31.213760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:31.213760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:31.213760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:31.215339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:31.215339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:31.215339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:31.215339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:31.215339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:31.215339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:31.215339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:31.215339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:31.215339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:31.215339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:31.215339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:31.215339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:31.215339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:31.215339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:31.215339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:31.215339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:31.216493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:31.216493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:31.216493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:31.216493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:31.216493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:31.216493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:31.216493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:31.216493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:31.216493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:31.216493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:31.216493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:31.216493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:31.216493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:31.216493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 11:26:31.216493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:31.224072 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 442/858] 10.222.0.13 () {56 vars in 1132 bytes} [Thu Jun 23 11:26:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:31.225088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e883a08-8234-48b6-be30-33c2a4de9f76 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:31.265999 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:31.308720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271df53-91d1-4a0e-9058-107b5890846b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:31.334184 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e883a08-8234-48b6-be30-33c2a4de9f76 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:31.355133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915ae83-cdcb-43f3-b941-e0c0fdebd8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:31.355980 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915ae83-cdcb-43f3-b941-e0c0fdebd8ed None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:31.356346 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915ae83-cdcb-43f3-b941-e0c0fdebd8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:31.356626 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915ae83-cdcb-43f3-b941-e0c0fdebd8ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:31.371933 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915ae83-cdcb-43f3-b941-e0c0fdebd8ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:31.372857 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f678609-4117-4864-a077-1263fd694e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:31.373407 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915ae83-cdcb-43f3-b941-e0c0fdebd8ed None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:31.374150 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f678609-4117-4864-a077-1263fd694e8b None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:31.374766 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f678609-4117-4864-a077-1263fd694e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:31.375254 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f678609-4117-4864-a077-1263fd694e8b None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:31.376221 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f678609-4117-4864-a077-1263fd694e8b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:31.376730 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f678609-4117-4864-a077-1263fd694e8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:31.378288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f678609-4117-4864-a077-1263fd694e8b None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:31.388608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 443/859] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 23 11:26:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:31.389767 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f678609-4117-4864-a077-1263fd694e8b None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:31.394310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-1915ae83-cdcb-43f3-b941-e0c0fdebd8ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:31.394310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:31.394310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:31.394310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:31.394310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:31.394310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:31.394310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:31.394310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:31.394310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:31.394310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:31.394310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:31.394310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:31.394310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:31.394310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:31.394310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:31.395235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:31.395235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:31.395235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:31.395235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:31.395235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:31.395235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:31.395235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:31.395235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:31.395235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:31.395235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:31.395235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:31.395235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:31.395235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:31.395235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:31.395235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:31.395235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:31.395235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:31.396138 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:31.396138 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:31.396138 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:31.396138 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:31.396138 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 417/860] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:31.401296 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915ae83-cdcb-43f3-b941-e0c0fdebd8ed None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:31.462400 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:31.520027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915ae83-cdcb-43f3-b941-e0c0fdebd8ed None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:31.526170 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f678609-4117-4864-a077-1263fd694e8b None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:31.540385 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:31.560161 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735c073-66a0-4836-9948-8be3aa255e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:31.560749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735c073-66a0-4836-9948-8be3aa255e5f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:31.561032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735c073-66a0-4836-9948-8be3aa255e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:31.561307 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735c073-66a0-4836-9948-8be3aa255e5f None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:31.562038 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735c073-66a0-4836-9948-8be3aa255e5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:31.562354 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735c073-66a0-4836-9948-8be3aa255e5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:31.563806 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735c073-66a0-4836-9948-8be3aa255e5f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:31.564953 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e6907-3c35-40c6-a5c8-331c4fe30717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:31.565625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e6907-3c35-40c6-a5c8-331c4fe30717 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:31.565929 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e6907-3c35-40c6-a5c8-331c4fe30717 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:31.566492 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e6907-3c35-40c6-a5c8-331c4fe30717 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:31.566947 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e6907-3c35-40c6-a5c8-331c4fe30717 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:31.567511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e6907-3c35-40c6-a5c8-331c4fe30717 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:31.568865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e6907-3c35-40c6-a5c8-331c4fe30717 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:31.575698 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 444/861] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 23 11:26:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:31.577227 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b65602-1f26-4169-b42d-8dd39d4d947e None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:31.577458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 418/862] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:31.578026 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b65602-1f26-4169-b42d-8dd39d4d947e None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:31.578303 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b65602-1f26-4169-b42d-8dd39d4d947e None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:31.579704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 445/863] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:31.584193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735c073-66a0-4836-9948-8be3aa255e5f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:31.584993 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8576b-d01e-4ff1-ade6-f3fbc1d73fb1 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:31.585285 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8576b-d01e-4ff1-ade6-f3fbc1d73fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:31.585572 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8576b-d01e-4ff1-ade6-f3fbc1d73fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:31.606033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-54e8576b-d01e-4ff1-ade6-f3fbc1d73fb1 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:31.607415 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-54e8576b-d01e-4ff1-ade6-f3fbc1d73fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:31.692967 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 446/864] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:31.718809 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735c073-66a0-4836-9948-8be3aa255e5f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:31.721642 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8576b-d01e-4ff1-ade6-f3fbc1d73fb1 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:31.764824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8bbfa-a995-4667-bca1-9b6ca4b99346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:31.765640 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8bbfa-a995-4667-bca1-9b6ca4b99346 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:31.766006 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8bbfa-a995-4667-bca1-9b6ca4b99346 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:31.766284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8bbfa-a995-4667-bca1-9b6ca4b99346 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:31.770777 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:31.774129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8bbfa-a995-4667-bca1-9b6ca4b99346 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:31.775638 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8bbfa-a995-4667-bca1-9b6ca4b99346 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:31.792486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-94a8bbfa-a995-4667-bca1-9b6ca4b99346 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:26:31.792486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:31.792486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:31.792486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:31.792486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:31.792486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:31.792486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:31.792486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:31.792486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:31.792486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:31.792486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:31.792486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:31.792486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:31.792486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:31.792486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:31.792486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:31.793801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:31.793801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:31.793801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:31.793801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:31.793801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:31.793801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:31.793801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:31.793801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:31.793801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:31.793801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:31.793801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:31.793801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:31.793801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:31.793801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:31.793801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:31.793801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:31.793801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:31.793801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:31.794885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:31.794885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:31.794885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:31.794885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:31.794885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:31.794885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:31.794885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:31.794885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:31.794885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:26:31.794885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:31.794885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 419/865] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jun 23 11:26:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:31.799569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8bbfa-a995-4667-bca1-9b6ca4b99346 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:31.836713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8576b-d01e-4ff1-ade6-f3fbc1d73fb1 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:31.843664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:31.883432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf734f5-b495-4d76-a438-0d6b7f95a947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:31.884981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf734f5-b495-4d76-a438-0d6b7f95a947 None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:31.885254 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf734f5-b495-4d76-a438-0d6b7f95a947 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:31.885531 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf734f5-b495-4d76-a438-0d6b7f95a947 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:31.886271 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf734f5-b495-4d76-a438-0d6b7f95a947 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:31.888302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf734f5-b495-4d76-a438-0d6b7f95a947 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:31.915811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8bbfa-a995-4667-bca1-9b6ca4b99346 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:31.953450 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c77ac-e206-4485-8bab-37c7b9534c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:31.954125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c77ac-e206-4485-8bab-37c7b9534c31 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:31.954516 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c77ac-e206-4485-8bab-37c7b9534c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:31.956375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c77ac-e206-4485-8bab-37c7b9534c31 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:31.962004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 447/866] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 23 11:26:31 2022] POST /identity/v3/services => generated 251 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:26:31.967185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c77ac-e206-4485-8bab-37c7b9534c31 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:31.967185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c77ac-e206-4485-8bab-37c7b9534c31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:31.967185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c77ac-e206-4485-8bab-37c7b9534c31 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:31.989646 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 420/867] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Jun 23 11:26:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:31.995557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf734f5-b495-4d76-a438-0d6b7f95a947 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:32.117765 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:32.125978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf734f5-b495-4d76-a438-0d6b7f95a947 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:32.166688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24633c47-b06c-4b4c-a3ca-316669e14553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:32.167350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24633c47-b06c-4b4c-a3ca-316669e14553 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:32.167632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24633c47-b06c-4b4c-a3ca-316669e14553 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:32.167947 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24633c47-b06c-4b4c-a3ca-316669e14553 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:32.240429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24633c47-b06c-4b4c-a3ca-316669e14553 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:32.241958 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24633c47-b06c-4b4c-a3ca-316669e14553 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:32.340435 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:32.349337 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 448/868] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:26:31 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 11:26:32.354580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c77ac-e206-4485-8bab-37c7b9534c31 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:32.466939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:32.473841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c77ac-e206-4485-8bab-37c7b9534c31 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:32.494001 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42bf83-2444-4d2f-8c59-8c6d5da5393f None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:32.494291 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42bf83-2444-4d2f-8c59-8c6d5da5393f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:32.494588 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42bf83-2444-4d2f-8c59-8c6d5da5393f None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:32.496280 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 449/869] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:26:32.501402 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bda4d-3e60-4a3e-b3f3-2ad7598f9aea None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:32.501690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bda4d-3e60-4a3e-b3f3-2ad7598f9aea None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:32.501972 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bda4d-3e60-4a3e-b3f3-2ad7598f9aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:32.518951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b226e-7ee7-4603-9bcc-04a6e0a5c413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:32.519768 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b226e-7ee7-4603-9bcc-04a6e0a5c413 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:32.520059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b226e-7ee7-4603-9bcc-04a6e0a5c413 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:32.520336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b226e-7ee7-4603-9bcc-04a6e0a5c413 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:32.520786 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c23bda4d-3e60-4a3e-b3f3-2ad7598f9aea None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:32.522019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c23bda4d-3e60-4a3e-b3f3-2ad7598f9aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:32.533548 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b226e-7ee7-4603-9bcc-04a6e0a5c413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:32.535732 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b226e-7ee7-4603-9bcc-04a6e0a5c413 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:32.559880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-bf8b226e-7ee7-4603-9bcc-04a6e0a5c413 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:32.559880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:32.559880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:32.559880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:32.559880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:32.559880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:32.559880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:32.559880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:32.559880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:32.559880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:32.559880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:32.559880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:32.559880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:32.559880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:32.559880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:32.560870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:32.560870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:32.560870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:32.560870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:32.560870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:32.560870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:32.560870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:32.560870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:32.560870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:32.560870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:32.560870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:32.560870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:32.560870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:32.560870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:32.560870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:32.560870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:32.560870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:32.561754 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:32.561754 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:32.561754 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:32.561754 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:32.566553 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 421/870] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:32.568221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b226e-7ee7-4603-9bcc-04a6e0a5c413 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:32.608034 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 450/871] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:32 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:26:32.630553 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bda4d-3e60-4a3e-b3f3-2ad7598f9aea None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:32.688878 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b226e-7ee7-4603-9bcc-04a6e0a5c413 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:32.744004 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ef895-20a1-41aa-840d-0d565bebda8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:32.744580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ef895-20a1-41aa-840d-0d565bebda8c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:32.745001 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ef895-20a1-41aa-840d-0d565bebda8c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:32.745404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ef895-20a1-41aa-840d-0d565bebda8c None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:32.746424 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ef895-20a1-41aa-840d-0d565bebda8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:32.747109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ef895-20a1-41aa-840d-0d565bebda8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:32.749163 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ef895-20a1-41aa-840d-0d565bebda8c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:32.764990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bda4d-3e60-4a3e-b3f3-2ad7598f9aea None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:32.771981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 422/872] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:32.773414 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ef895-20a1-41aa-840d-0d565bebda8c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:32.786362 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:32.803711 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:32.811736 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1e386-9c8a-4f7e-b4f4-0306c0c19c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:32.812761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1e386-9c8a-4f7e-b4f4-0306c0c19c25 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:32.813050 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1e386-9c8a-4f7e-b4f4-0306c0c19c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:32.813329 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1e386-9c8a-4f7e-b4f4-0306c0c19c25 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:32.813992 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f1e386-9c8a-4f7e-b4f4-0306c0c19c25 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:32.815427 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f1e386-9c8a-4f7e-b4f4-0306c0c19c25 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:32.828825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-16f1e386-9c8a-4f7e-b4f4-0306c0c19c25 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:26:32.828825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:32.828825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:32.828825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:32.828825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:32.828825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:32.828825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:32.828825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:32.828825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:26:32.828825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:32.828825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:32.828825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:32.828825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:32.828825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:32.828825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:32.828825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:32.828825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:32.830126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:32.830126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:32.830126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:32.830126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:32.830126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:32.830126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:32.830126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:32.830126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:32.830126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:32.830126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:32.830126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:32.830126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:32.830126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:32.830126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:32.830126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:32.830126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:32.831022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:32.831022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:32.831022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:32.831022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:32.831022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:32.831022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:32.831022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:32.831022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:32.831022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:32.831022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:32.831022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:32.831022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:32.831022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:26:32.831022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:32.834873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 451/873] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 23 11:26:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:32.836290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1e386-9c8a-4f7e-b4f4-0306c0c19c25 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:32.911738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ef895-20a1-41aa-840d-0d565bebda8c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:32.948877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357244e-385e-429a-a622-39277ac3c149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:32.950591 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357244e-385e-429a-a622-39277ac3c149 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:32.951097 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357244e-385e-429a-a622-39277ac3c149 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:32.951637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357244e-385e-429a-a622-39277ac3c149 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:32.955333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1e386-9c8a-4f7e-b4f4-0306c0c19c25 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:32.959236 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357244e-385e-429a-a622-39277ac3c149 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:32.960911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357244e-385e-429a-a622-39277ac3c149 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:32.976751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-5357244e-385e-429a-a622-39277ac3c149 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:32.976751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:32.976751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:32.976751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:32.976751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:32.976751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:32.976751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:32.976751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:32.976751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:32.976751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:32.976751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:32.976751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:32.976751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:32.976751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:32.976751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:32.977733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:32.977733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:32.977733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:32.977733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:32.977733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:32.977733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:32.977733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:32.977733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:32.977733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:32.977733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:32.977733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:32.977733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:32.977733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:32.977733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:32.977733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:32.977733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:32.977733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:32.978641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:32.978641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:32.978641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:32.978641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:32.978641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:32.978641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:32.978641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:32.978641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:32.985869 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 423/874] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:32.987273 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357244e-385e-429a-a622-39277ac3c149 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:32.990806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25b483-8489-42dc-a83c-126456934292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:32.991471 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25b483-8489-42dc-a83c-126456934292 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:32.991754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25b483-8489-42dc-a83c-126456934292 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:32.992037 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25b483-8489-42dc-a83c-126456934292 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:32.993194 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25b483-8489-42dc-a83c-126456934292 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:32.993194 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25b483-8489-42dc-a83c-126456934292 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:32.994574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25b483-8489-42dc-a83c-126456934292 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:32.999982 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 452/875] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jun 23 11:26:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:33.006520 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25b483-8489-42dc-a83c-126456934292 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:33.063392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:33.076742 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:33.100591 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357244e-385e-429a-a622-39277ac3c149 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:33.122432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25b483-8489-42dc-a83c-126456934292 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:33.136344 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953e7a8-e183-4b33-b3e0-30da91e834a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:33.137956 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953e7a8-e183-4b33-b3e0-30da91e834a5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:33.138444 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953e7a8-e183-4b33-b3e0-30da91e834a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:33.138911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953e7a8-e183-4b33-b3e0-30da91e834a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:33.139853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953e7a8-e183-4b33-b3e0-30da91e834a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:33.140317 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953e7a8-e183-4b33-b3e0-30da91e834a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:33.141884 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953e7a8-e183-4b33-b3e0-30da91e834a5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:33.160153 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 424/876] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:33.161672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953e7a8-e183-4b33-b3e0-30da91e834a5 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:33.163468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d45719-e381-4909-80a1-1d3510e9f15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:33.164212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d45719-e381-4909-80a1-1d3510e9f15e None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:33.164565 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d45719-e381-4909-80a1-1d3510e9f15e None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:33.164915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d45719-e381-4909-80a1-1d3510e9f15e None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:33.165969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d45719-e381-4909-80a1-1d3510e9f15e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:33.167276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d45719-e381-4909-80a1-1d3510e9f15e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:33.168970 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d45719-e381-4909-80a1-1d3510e9f15e None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:33.175333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 453/877] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jun 23 11:26:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:33.290474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953e7a8-e183-4b33-b3e0-30da91e834a5 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:33.320909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:33.331312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ece1a-37b4-4512-a096-45fedabaeed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:33.332288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ece1a-37b4-4512-a096-45fedabaeed3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:33.332718 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ece1a-37b4-4512-a096-45fedabaeed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:33.333146 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ece1a-37b4-4512-a096-45fedabaeed3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:33.334146 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ece1a-37b4-4512-a096-45fedabaeed3 None admin] RBAC: Adding query filter params (role.id=13708a4819a94b94b47ba5994aaa9310, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=030614ef56f143f8be6fcc0445ec9568) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:33.334643 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ece1a-37b4-4512-a096-45fedabaeed3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:33.336230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ece1a-37b4-4512-a096-45fedabaeed3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:33.342839 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 425/878] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:26:33 2022] GET /identity/v3/role_assignments?user.id=030614ef56f143f8be6fcc0445ec9568&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=13708a4819a94b94b47ba5994aaa9310 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:33.847442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb45a9f-2982-4cb1-bbca-7c0ecb5b1bae None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:33.847871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb45a9f-2982-4cb1-bbca-7c0ecb5b1bae None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:33.848051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb45a9f-2982-4cb1-bbca-7c0ecb5b1bae None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:33.849692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 454/879] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:33.855857 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315dc44-de49-4271-adc2-a2f09a1db7c2 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:33.856436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315dc44-de49-4271-adc2-a2f09a1db7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:33.856436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315dc44-de49-4271-adc2-a2f09a1db7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:33.874145 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-1315dc44-de49-4271-adc2-a2f09a1db7c2 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:33.875369 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1315dc44-de49-4271-adc2-a2f09a1db7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:33.947676 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 426/880] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:33.972386 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb45a9f-2982-4cb1-bbca-7c0ecb5b1bae None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:34.081115 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb45a9f-2982-4cb1-bbca-7c0ecb5b1bae None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:34.101738 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:34.116144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6beeb-d60e-4403-a4f8-3933affa80b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:34.116812 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6beeb-d60e-4403-a4f8-3933affa80b3 None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:34.117098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6beeb-d60e-4403-a4f8-3933affa80b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:34.119811 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6beeb-d60e-4403-a4f8-3933affa80b3 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:34.119811 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6beeb-d60e-4403-a4f8-3933affa80b3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:34.119811 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6beeb-d60e-4403-a4f8-3933affa80b3 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:34.165166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935aeb4b-c710-4a7e-b438-5c3b8e3696bd None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:34.165405 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935aeb4b-c710-4a7e-b438-5c3b8e3696bd None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:34.165687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935aeb4b-c710-4a7e-b438-5c3b8e3696bd None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:34.170565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 427/881] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:34.173601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971489a8-e981-4df8-93c2-2732327ee2e7 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:34.174094 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971489a8-e981-4df8-93c2-2732327ee2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:34.174694 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971489a8-e981-4df8-93c2-2732327ee2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:34.193012 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-971489a8-e981-4df8-93c2-2732327ee2e7 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:34.194610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-971489a8-e981-4df8-93c2-2732327ee2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:34.210525 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 455/882] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 23 11:26:33 2022] POST /identity/v3/services => generated 269 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:26:34.290117 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 428/883] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:34.313588 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6beeb-d60e-4403-a4f8-3933affa80b3 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:34.428527 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6beeb-d60e-4403-a4f8-3933affa80b3 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:34.465421 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015aa5d-c4b5-422e-912d-edd6cd45cc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:34.466221 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015aa5d-c4b5-422e-912d-edd6cd45cc44 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:34.466499 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015aa5d-c4b5-422e-912d-edd6cd45cc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:34.466790 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015aa5d-c4b5-422e-912d-edd6cd45cc44 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:34.467446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015aa5d-c4b5-422e-912d-edd6cd45cc44 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:34.468804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015aa5d-c4b5-422e-912d-edd6cd45cc44 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:34.480724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-3015aa5d-c4b5-422e-912d-edd6cd45cc44 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 11:26:34.480724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:34.480724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:34.480724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:34.480724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:34.480724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:34.480724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:34.480724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:34.480724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 11:26:34.480724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:34.480724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:34.480724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:34.480724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:34.480724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:34.480724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:34.480724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:34.480724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:34.481867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:34.481867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:34.481867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:34.481867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:34.481867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:34.481867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:34.481867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:34.481867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:34.481867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:34.481867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:34.481867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:34.481867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:34.481867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:34.481867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:34.481867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:34.481867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:34.482750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:34.482750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:34.482750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:34.482750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:34.482750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:34.482750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:34.482750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:34.482750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:34.482750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:34.482750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:34.482750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:34.482750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:34.482750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 11:26:34.482750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:34.482750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 456/884] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:26:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:34.488527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971489a8-e981-4df8-93c2-2732327ee2e7 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:34.541936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:34.602083 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971489a8-e981-4df8-93c2-2732327ee2e7 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:34.638941 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f882c-764b-4a3f-bc15-d81e257b5215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:34.639722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f882c-764b-4a3f-bc15-d81e257b5215 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:34.640010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f882c-764b-4a3f-bc15-d81e257b5215 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:34.640289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f882c-764b-4a3f-bc15-d81e257b5215 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:34.641003 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f882c-764b-4a3f-bc15-d81e257b5215 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:34.641309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f882c-764b-4a3f-bc15-d81e257b5215 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:34.642708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f882c-764b-4a3f-bc15-d81e257b5215 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:34.649125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 429/885] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:26:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:34.655097 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015aa5d-c4b5-422e-912d-edd6cd45cc44 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:34.703153 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:34.760867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015aa5d-c4b5-422e-912d-edd6cd45cc44 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:34.801848 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c19ed-3c8a-48c1-8219-f46a833abc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:34.802891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c19ed-3c8a-48c1-8219-f46a833abc3f None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:34.803411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c19ed-3c8a-48c1-8219-f46a833abc3f None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:34.803902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c19ed-3c8a-48c1-8219-f46a833abc3f None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:34.804832 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c19ed-3c8a-48c1-8219-f46a833abc3f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:34.805357 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c19ed-3c8a-48c1-8219-f46a833abc3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:34.806999 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c19ed-3c8a-48c1-8219-f46a833abc3f None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:34.813557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 457/886] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:26:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:34.819402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f882c-764b-4a3f-bc15-d81e257b5215 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:34.929445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f882c-764b-4a3f-bc15-d81e257b5215 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:34.941017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:34.967152 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a0456-f61d-4e44-be5f-4c7f4360f818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:34.967738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a0456-f61d-4e44-be5f-4c7f4360f818 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:34.968021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a0456-f61d-4e44-be5f-4c7f4360f818 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:34.968297 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a0456-f61d-4e44-be5f-4c7f4360f818 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:34.969093 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a0456-f61d-4e44-be5f-4c7f4360f818 None admin] RBAC: Adding query filter params (interface=public, service_id=189379c695994b02858231972d171b5d, region_id=RegionOne) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:34.969404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a0456-f61d-4e44-be5f-4c7f4360f818 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:34.970806 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a0456-f61d-4e44-be5f-4c7f4360f818 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:34.976388 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 430/887] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jun 23 11:26:34 2022] GET /identity/v3/endpoints?service_id=189379c695994b02858231972d171b5d&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:34.982155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c19ed-3c8a-48c1-8219-f46a833abc3f None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:35.087945 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c19ed-3c8a-48c1-8219-f46a833abc3f None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:35.097737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:35.123070 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8fb1d-999c-4062-b79b-7e416c7f8b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:35.123732 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8fb1d-999c-4062-b79b-7e416c7f8b90 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:35.124020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8fb1d-999c-4062-b79b-7e416c7f8b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:35.124299 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8fb1d-999c-4062-b79b-7e416c7f8b90 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:35.124984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8fb1d-999c-4062-b79b-7e416c7f8b90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:35.126350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8fb1d-999c-4062-b79b-7e416c7f8b90 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:35.132458 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 458/888] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jun 23 11:26:34 2022] GET /identity/v3/services => generated 1268 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:26:35.334062 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f4a19-cbe5-4677-9cfb-03ee2771ea6b None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:35.334450 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f4a19-cbe5-4677-9cfb-03ee2771ea6b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:35.334692 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f4a19-cbe5-4677-9cfb-03ee2771ea6b None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:35.336276 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 431/889] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:35.342305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fb1c8-ea9e-4600-aadf-dceeddd652ad None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:35.342623 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fb1c8-ea9e-4600-aadf-dceeddd652ad None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:35.342909 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fb1c8-ea9e-4600-aadf-dceeddd652ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:35.364839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-9b0fb1c8-ea9e-4600-aadf-dceeddd652ad None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:35.366020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-9b0fb1c8-ea9e-4600-aadf-dceeddd652ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:35.446203 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 459/890] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:35.469217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f4a19-cbe5-4677-9cfb-03ee2771ea6b None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:35.534739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138f8be-b278-4207-a9c3-e827f2163726 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:35.535076 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138f8be-b278-4207-a9c3-e827f2163726 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:35.535309 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138f8be-b278-4207-a9c3-e827f2163726 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:35.537064 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 460/891] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:35.542044 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aefbcd-a992-434b-8707-6ef4812d3bb7 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:35.542341 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aefbcd-a992-434b-8707-6ef4812d3bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:35.542719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aefbcd-a992-434b-8707-6ef4812d3bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:35.560700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-39aefbcd-a992-434b-8707-6ef4812d3bb7 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:35.561809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-39aefbcd-a992-434b-8707-6ef4812d3bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:35.586656 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f4a19-cbe5-4677-9cfb-03ee2771ea6b None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:35.632271 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebdfa0-9162-4b67-a6df-07137f97600a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:35.633915 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebdfa0-9162-4b67-a6df-07137f97600a None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:35.634398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebdfa0-9162-4b67-a6df-07137f97600a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:35.635148 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebdfa0-9162-4b67-a6df-07137f97600a None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:35.637067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebdfa0-9162-4b67-a6df-07137f97600a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:35.639152 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebdfa0-9162-4b67-a6df-07137f97600a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:35.644462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 461/892] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:26:35.668173 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aefbcd-a992-434b-8707-6ef4812d3bb7 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:35.700266 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:35.726661 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 432/893] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 23 11:26:35 2022] POST /identity/v3/services => generated 241 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:26:35.783522 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aefbcd-a992-434b-8707-6ef4812d3bb7 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:35.870211 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082ba01-9cdc-41e9-9a35-fd76fba241e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:35.871555 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082ba01-9cdc-41e9-9a35-fd76fba241e5 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:35.871888 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082ba01-9cdc-41e9-9a35-fd76fba241e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:35.872110 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082ba01-9cdc-41e9-9a35-fd76fba241e5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:35.884150 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082ba01-9cdc-41e9-9a35-fd76fba241e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:35.892570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-4082ba01-9cdc-41e9-9a35-fd76fba241e5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:26:35.892570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:35.892570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:35.892570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:35.892570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:35.892570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:35.892570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:35.892570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:35.892570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:35.892570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:35.892570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:35.892570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:35.892570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:35.892570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:35.892570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:35.892570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:35.893514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:35.893514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:35.893514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:35.893514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:35.893514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:35.893514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:35.893514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:35.893514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:35.893514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:35.893514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:35.893514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:35.893514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:35.893514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:35.893514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:35.893514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:35.893514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:35.894810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:35.894810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:35.894810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:35.894810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:35.894810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:35.894810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:35.894810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:35.894810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:35.894810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:35.894810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:35.894810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:26:35.894810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:35.894810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 462/894] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Jun 23 11:26:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:35.900141 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebdfa0-9162-4b67-a6df-07137f97600a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:35.939801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:36.013016 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebdfa0-9162-4b67-a6df-07137f97600a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:36.048647 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8f8ca-7eb8-4551-b6e3-7d8dc8143760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:36.049334 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8f8ca-7eb8-4551-b6e3-7d8dc8143760 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:36.049622 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8f8ca-7eb8-4551-b6e3-7d8dc8143760 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:36.049895 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8f8ca-7eb8-4551-b6e3-7d8dc8143760 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:36.050671 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8f8ca-7eb8-4551-b6e3-7d8dc8143760 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:36.050982 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8f8ca-7eb8-4551-b6e3-7d8dc8143760 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:36.052344 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8f8ca-7eb8-4551-b6e3-7d8dc8143760 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:36.061914 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 433/895] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jun 23 11:26:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:36.067620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082ba01-9cdc-41e9-9a35-fd76fba241e5 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:36.160227 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:36.172242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082ba01-9cdc-41e9-9a35-fd76fba241e5 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:36.207932 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44c338-9ae3-486f-aed5-b9ab6ad7f9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:36.208735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44c338-9ae3-486f-aed5-b9ab6ad7f9da None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:36.209018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44c338-9ae3-486f-aed5-b9ab6ad7f9da None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:36.209295 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44c338-9ae3-486f-aed5-b9ab6ad7f9da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:36.220394 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44c338-9ae3-486f-aed5-b9ab6ad7f9da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:36.221907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44c338-9ae3-486f-aed5-b9ab6ad7f9da None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:36.239852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-2b44c338-9ae3-486f-aed5-b9ab6ad7f9da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:36.239852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:36.239852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:36.239852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:36.239852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:36.239852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:36.239852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:36.239852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:36.239852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:36.239852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:36.239852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:36.239852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:36.239852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:36.239852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:36.239852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:36.240829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:36.240829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:36.240829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:36.240829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:36.240829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:36.240829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:36.240829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:36.240829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:36.240829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:36.240829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:36.240829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:36.240829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:36.240829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:36.240829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:36.240829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:36.240829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:36.240829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:36.242156 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:36.242156 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:36.242156 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:36.242156 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:36.242156 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 463/896] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:36.247553 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8f8ca-7eb8-4551-b6e3-7d8dc8143760 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:36.347619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:36.353517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8f8ca-7eb8-4551-b6e3-7d8dc8143760 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:36.389373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e613f-dc3a-4a2d-b247-5e010d4c3bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:36.390046 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e613f-dc3a-4a2d-b247-5e010d4c3bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:36.390333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e613f-dc3a-4a2d-b247-5e010d4c3bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:36.390634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e613f-dc3a-4a2d-b247-5e010d4c3bd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:36.391420 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e613f-dc3a-4a2d-b247-5e010d4c3bd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:36.391734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e613f-dc3a-4a2d-b247-5e010d4c3bd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:36.393100 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e613f-dc3a-4a2d-b247-5e010d4c3bd2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:36.405580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 434/897] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:36.411100 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44c338-9ae3-486f-aed5-b9ab6ad7f9da None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:36.455666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f03e3f-cd48-4793-a40b-2cc003532212 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:36.456013 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f03e3f-cd48-4793-a40b-2cc003532212 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:36.456597 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f03e3f-cd48-4793-a40b-2cc003532212 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:36.458907 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 435/898] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:26:36.463230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c381ac4-8ef7-417e-8d94-13ff61f18b84 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:36.463515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c381ac4-8ef7-417e-8d94-13ff61f18b84 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:36.463793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c381ac4-8ef7-417e-8d94-13ff61f18b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:36.485017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3c381ac4-8ef7-417e-8d94-13ff61f18b84 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:36.486242 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3c381ac4-8ef7-417e-8d94-13ff61f18b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:36.519493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44c338-9ae3-486f-aed5-b9ab6ad7f9da None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:36.554400 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bca1dc-b644-4d4a-b8bb-ff42197edc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:36.555240 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bca1dc-b644-4d4a-b8bb-ff42197edc48 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:36.555525 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bca1dc-b644-4d4a-b8bb-ff42197edc48 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:36.555803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bca1dc-b644-4d4a-b8bb-ff42197edc48 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:36.563716 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bca1dc-b644-4d4a-b8bb-ff42197edc48 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:36.565165 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bca1dc-b644-4d4a-b8bb-ff42197edc48 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:36.565549 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 436/899] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:26:36.572890 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:36.586136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-b5bca1dc-b644-4d4a-b8bb-ff42197edc48 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:26:36.586136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:36.586136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:36.586136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:36.586136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:36.586136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:36.586136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:36.586136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:36.586136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:36.586136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:36.586136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:36.586136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:36.586136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:36.586136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:36.586136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:36.586136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:36.596242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:36.596242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:36.596242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:36.596242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:36.596242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:36.596242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:36.596242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:36.596242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:36.596242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:36.596242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:36.596242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:36.596242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:36.596242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:36.596242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:36.596242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:36.596242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:36.596242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:36.596242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:36.597220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:36.597220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:36.597220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:36.597220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:36.597220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:36.597220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:36.597220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:36.597220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:36.597220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:26:36.597220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:36.597220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 464/900] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jun 23 11:26:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:36.597813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c381ac4-8ef7-417e-8d94-13ff61f18b84 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:36.597902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bca1dc-b644-4d4a-b8bb-ff42197edc48 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:36.698653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:36.715600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bca1dc-b644-4d4a-b8bb-ff42197edc48 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:36.717761 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c381ac4-8ef7-417e-8d94-13ff61f18b84 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:36.758359 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012ad11-3fb8-4855-bd69-fffd0580d2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:36.759473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012ad11-3fb8-4855-bd69-fffd0580d2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:36.759945 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012ad11-3fb8-4855-bd69-fffd0580d2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:36.760404 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012ad11-3fb8-4855-bd69-fffd0580d2f5 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:36.761568 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012ad11-3fb8-4855-bd69-fffd0580d2f5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:36.762069 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012ad11-3fb8-4855-bd69-fffd0580d2f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:36.763244 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf999e-6da2-48d3-af04-7a8487f21360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:36.763985 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012ad11-3fb8-4855-bd69-fffd0580d2f5 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:36.764095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf999e-6da2-48d3-af04-7a8487f21360 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:36.764330 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf999e-6da2-48d3-af04-7a8487f21360 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:36.764609 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf999e-6da2-48d3-af04-7a8487f21360 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:36.765272 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf999e-6da2-48d3-af04-7a8487f21360 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:36.766798 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf999e-6da2-48d3-af04-7a8487f21360 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:36.780000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-22cf999e-6da2-48d3-af04-7a8487f21360 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 11:26:36.780000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:36.780000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:36.780000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:36.780000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:36.780000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:36.780000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:36.780000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:36.780000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 11:26:36.780000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:36.780000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:36.780000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:36.780000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:36.780000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:36.780000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:36.780000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:36.780985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:36.780985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:36.780985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:36.780985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:36.780985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:36.780985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:36.780985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:36.780985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:36.780985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:36.780985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:36.780985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:36.780985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:36.780985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:36.780985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:36.780985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:36.780985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:36.781826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:36.781826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:36.781826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:36.781826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:36.781826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:36.781826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:36.781826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:36.781826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:36.781826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:36.781826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:36.781826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:36.781826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:36.781826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:36.781826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 11:26:36.781826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:36.781826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 437/901] 10.222.0.13 () {56 vars in 1132 bytes} [Thu Jun 23 11:26:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:36.788365 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf999e-6da2-48d3-af04-7a8487f21360 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:36.791865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 465/902] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Jun 23 11:26:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:36.797637 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012ad11-3fb8-4855-bd69-fffd0580d2f5 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:36.863251 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:36.906003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012ad11-3fb8-4855-bd69-fffd0580d2f5 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:36.929520 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf999e-6da2-48d3-af04-7a8487f21360 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:36.942050 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae6c42-a61a-411b-bd12-7594e25b3afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:36.942972 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae6c42-a61a-411b-bd12-7594e25b3afe None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:36.943252 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae6c42-a61a-411b-bd12-7594e25b3afe None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:36.943524 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae6c42-a61a-411b-bd12-7594e25b3afe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:36.972249 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1340bd2-d62b-4b0e-8dc2-78e8ee9dd5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:36.973624 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1340bd2-d62b-4b0e-8dc2-78e8ee9dd5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:36.974107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1340bd2-d62b-4b0e-8dc2-78e8ee9dd5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:36.974694 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1340bd2-d62b-4b0e-8dc2-78e8ee9dd5f2 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:36.975707 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1340bd2-d62b-4b0e-8dc2-78e8ee9dd5f2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:36.976212 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1340bd2-d62b-4b0e-8dc2-78e8ee9dd5f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:36.977769 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1340bd2-d62b-4b0e-8dc2-78e8ee9dd5f2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:36.988550 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 438/903] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 23 11:26:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:36.989715 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1340bd2-d62b-4b0e-8dc2-78e8ee9dd5f2 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:37.007764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:37.013135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae6c42-a61a-411b-bd12-7594e25b3afe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:37.014645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae6c42-a61a-411b-bd12-7594e25b3afe None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:37.072363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:37.105732 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1340bd2-d62b-4b0e-8dc2-78e8ee9dd5f2 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:37.115751 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 466/904] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:26:36 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:26:37.120975 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae6c42-a61a-411b-bd12-7594e25b3afe None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:37.148882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2647da-f4a5-45db-a2ac-db7e714c2586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:37.149944 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2647da-f4a5-45db-a2ac-db7e714c2586 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:37.150433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2647da-f4a5-45db-a2ac-db7e714c2586 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:37.150968 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2647da-f4a5-45db-a2ac-db7e714c2586 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:37.152392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2647da-f4a5-45db-a2ac-db7e714c2586 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:37.152885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2647da-f4a5-45db-a2ac-db7e714c2586 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:37.154465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2647da-f4a5-45db-a2ac-db7e714c2586 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:37.164984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 439/905] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 23 11:26:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:37.166095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2647da-f4a5-45db-a2ac-db7e714c2586 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:37.223613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae6c42-a61a-411b-bd12-7594e25b3afe None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:37.259000 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4146f-174a-40b3-a96e-3e738dc3431e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:37.259608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4146f-174a-40b3-a96e-3e738dc3431e None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:37.259955 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4146f-174a-40b3-a96e-3e738dc3431e None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:37.260299 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4146f-174a-40b3-a96e-3e738dc3431e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:37.264568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:37.271257 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc4146f-174a-40b3-a96e-3e738dc3431e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:37.272788 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc4146f-174a-40b3-a96e-3e738dc3431e None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:37.289489 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2647da-f4a5-45db-a2ac-db7e714c2586 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:37.291350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-adc4146f-174a-40b3-a96e-3e738dc3431e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:37.291350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:37.291350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:37.291350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:37.291350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:37.291350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:37.291350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:37.291350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:37.291350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:37.291350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:37.291350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:37.291350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:37.291350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:37.291350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:37.291350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:37.292238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:37.292238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:37.292238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:37.292238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:37.292238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:37.292238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:37.292238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:37.292238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:37.292238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:37.292238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:37.292238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:37.292238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:37.292238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:37.292238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:37.292238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:37.292238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:37.292238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:37.293179 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:37.293179 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:37.293179 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:37.293179 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:37.293179 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 467/906] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:37.299498 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4146f-174a-40b3-a96e-3e738dc3431e None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:37.312257 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:37.332505 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31737b-c108-4b33-a645-64e92e673782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:37.335272 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31737b-c108-4b33-a645-64e92e673782 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:37.335272 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31737b-c108-4b33-a645-64e92e673782 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:37.335272 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31737b-c108-4b33-a645-64e92e673782 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:37.335272 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31737b-c108-4b33-a645-64e92e673782 None admin] RBAC: Adding query filter params (interface=public, service_id=90f59e6498684b40aee3c454010166b6, region_id=RegionOne) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:37.335557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31737b-c108-4b33-a645-64e92e673782 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:37.337088 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31737b-c108-4b33-a645-64e92e673782 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:37.348724 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 440/907] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jun 23 11:26:37 2022] GET /identity/v3/endpoints?service_id=90f59e6498684b40aee3c454010166b6&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:37.348724 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5ef48-2d83-4b04-8832-4beafe6943fa None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:37.348724 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5ef48-2d83-4b04-8832-4beafe6943fa None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:37.348724 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5ef48-2d83-4b04-8832-4beafe6943fa None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:37.350613 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 441/908] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:37.351571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5ef48-2d83-4b04-8832-4beafe6943fa None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:37.410621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4146f-174a-40b3-a96e-3e738dc3431e None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:37.449968 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb77791-138f-4420-90e5-8bd1d4aa2bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:37.450769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb77791-138f-4420-90e5-8bd1d4aa2bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:37.451054 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb77791-138f-4420-90e5-8bd1d4aa2bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:37.451512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb77791-138f-4420-90e5-8bd1d4aa2bc1 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:37.452165 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb77791-138f-4420-90e5-8bd1d4aa2bc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:37.452475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb77791-138f-4420-90e5-8bd1d4aa2bc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:37.453978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb77791-138f-4420-90e5-8bd1d4aa2bc1 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:37.460589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5ef48-2d83-4b04-8832-4beafe6943fa None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:37.470897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 468/909] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:37.473354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90077498-4d84-410d-a978-271c0408a787 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:37.473710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90077498-4d84-410d-a978-271c0408a787 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:37.474198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90077498-4d84-410d-a978-271c0408a787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:37.491393 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-90077498-4d84-410d-a978-271c0408a787 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:37.492522 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-90077498-4d84-410d-a978-271c0408a787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:37.504018 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb46f3-c31d-4309-b024-6066ca6efa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:37.504733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb46f3-c31d-4309-b024-6066ca6efa55 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:37.505014 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb46f3-c31d-4309-b024-6066ca6efa55 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:37.505232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb46f3-c31d-4309-b024-6066ca6efa55 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:37.505914 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb46f3-c31d-4309-b024-6066ca6efa55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:37.507508 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb46f3-c31d-4309-b024-6066ca6efa55 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:37.514596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 442/910] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jun 23 11:26:37 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:26:37.515238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb46f3-c31d-4309-b024-6066ca6efa55 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:37.524641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:37.587708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:37.595432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 469/911] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:37.618255 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90077498-4d84-410d-a978-271c0408a787 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:37.644873 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb46f3-c31d-4309-b024-6066ca6efa55 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:37.686767 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe484b-c388-4b4b-b5a1-0778b5b2d2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:37.687844 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe484b-c388-4b4b-b5a1-0778b5b2d2ca None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:37.688127 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe484b-c388-4b4b-b5a1-0778b5b2d2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:37.688403 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe484b-c388-4b4b-b5a1-0778b5b2d2ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:37.701936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe484b-c388-4b4b-b5a1-0778b5b2d2ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:37.703440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe484b-c388-4b4b-b5a1-0778b5b2d2ca None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:37.719723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-26fe484b-c388-4b4b-b5a1-0778b5b2d2ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:37.719723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:37.719723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:37.719723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:37.719723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:37.719723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:37.719723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:37.719723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:37.719723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:37.719723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:37.719723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:37.719723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:37.719723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:37.719723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:37.719723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:37.720672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:37.720672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:37.720672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:37.720672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:37.720672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:37.720672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:37.720672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:37.720672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:37.720672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:37.720672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:37.720672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:37.720672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:37.720672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:37.720672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:37.720672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:37.720672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:37.720672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:37.721630 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:37.721630 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:37.721630 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:37.721630 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:37.721630 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:37.721630 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:37.721630 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:37.721630 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:37.721630 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 443/912] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:37.726597 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe484b-c388-4b4b-b5a1-0778b5b2d2ca None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:37.735521 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90077498-4d84-410d-a978-271c0408a787 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:37.783064 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f224afa-4a94-45ef-b511-8b6eedd4a3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:37.784287 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f224afa-4a94-45ef-b511-8b6eedd4a3b5 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:37.784673 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f224afa-4a94-45ef-b511-8b6eedd4a3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:37.785417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f224afa-4a94-45ef-b511-8b6eedd4a3b5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:37.786187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f224afa-4a94-45ef-b511-8b6eedd4a3b5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:37.787691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f224afa-4a94-45ef-b511-8b6eedd4a3b5 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:37.801933 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:37.805603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-1f224afa-4a94-45ef-b511-8b6eedd4a3b5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 11:26:37.805603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:37.805603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:37.805603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:37.805603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:37.805603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:37.805603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:37.805603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:37.805603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 11:26:37.805603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:37.805603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:37.805603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:37.805603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:37.805603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:37.805603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:37.805603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:37.805603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:37.806625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:37.806625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:37.806625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:37.806625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:37.806625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:37.806625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:37.806625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:37.806625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:37.806625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:37.806625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:37.806625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:37.806625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:37.806625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:37.806625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:37.806625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:37.806625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:37.807478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:37.807478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:37.807478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:37.807478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:37.807478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:37.807478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:37.807478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:37.807478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:37.807478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:37.807478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:37.807478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:37.807478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:37.807478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 11:26:37.807478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:37.808747 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 470/913] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:26:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:37.816207 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f224afa-4a94-45ef-b511-8b6eedd4a3b5 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:37.846381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe484b-c388-4b4b-b5a1-0778b5b2d2ca None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:37.883079 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:37.883493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0844e49-89bf-4d64-ba2a-3dcd2bac771f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:37.884508 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0844e49-89bf-4d64-ba2a-3dcd2bac771f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:37.884508 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0844e49-89bf-4d64-ba2a-3dcd2bac771f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:37.884916 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0844e49-89bf-4d64-ba2a-3dcd2bac771f None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:37.885769 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0844e49-89bf-4d64-ba2a-3dcd2bac771f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:37.886143 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0844e49-89bf-4d64-ba2a-3dcd2bac771f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:37.888280 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0844e49-89bf-4d64-ba2a-3dcd2bac771f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:37.900891 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 444/914] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:37.902256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb3849-29a8-4b8a-92af-132962b05b0c None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:37.902595 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb3849-29a8-4b8a-92af-132962b05b0c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:37.902860 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb3849-29a8-4b8a-92af-132962b05b0c None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:37.904241 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 445/915] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:37.907378 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb3849-29a8-4b8a-92af-132962b05b0c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:37.930148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f224afa-4a94-45ef-b511-8b6eedd4a3b5 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:37.969150 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6292a-d502-4593-8612-c6c63785bcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:37.970759 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6292a-d502-4593-8612-c6c63785bcac None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:37.971250 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6292a-d502-4593-8612-c6c63785bcac None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:37.971944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6292a-d502-4593-8612-c6c63785bcac None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:37.973211 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6292a-d502-4593-8612-c6c63785bcac None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:37.974049 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6292a-d502-4593-8612-c6c63785bcac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:37.975855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6292a-d502-4593-8612-c6c63785bcac None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:37.988115 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 471/916] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:26:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:37.988115 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d093029-327a-43b6-aae1-5279e03cab8d None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:37.988115 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d093029-327a-43b6-aae1-5279e03cab8d None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:37.988115 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d093029-327a-43b6-aae1-5279e03cab8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:38.006068 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-6d093029-327a-43b6-aae1-5279e03cab8d None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:38.007839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-6d093029-327a-43b6-aae1-5279e03cab8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:38.015984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb3849-29a8-4b8a-92af-132962b05b0c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:38.043310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:38.053408 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d619ad-0858-404f-9847-c2ddddbb9ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:38.054370 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d619ad-0858-404f-9847-c2ddddbb9ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:38.064949 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d619ad-0858-404f-9847-c2ddddbb9ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:38.064949 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d619ad-0858-404f-9847-c2ddddbb9ae8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:38.064949 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d619ad-0858-404f-9847-c2ddddbb9ae8 None admin] RBAC: Adding query filter params (role.id=a9a87bede27348b89fa09b9dfd04e89c, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=030614ef56f143f8be6fcc0445ec9568) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:38.064949 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d619ad-0858-404f-9847-c2ddddbb9ae8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:38.064949 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d619ad-0858-404f-9847-c2ddddbb9ae8 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:38.070745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 446/917] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:26:37 2022] GET /identity/v3/role_assignments?user.id=030614ef56f143f8be6fcc0445ec9568&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=a9a87bede27348b89fa09b9dfd04e89c => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:38.070745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d619ad-0858-404f-9847-c2ddddbb9ae8 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:38.086721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 472/918] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:38.112247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d093029-327a-43b6-aae1-5279e03cab8d None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:38.177839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:38.222388 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d093029-327a-43b6-aae1-5279e03cab8d None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:38.224086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d619ad-0858-404f-9847-c2ddddbb9ae8 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:38.273350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89069685-30d9-4814-9fb2-f645f1609410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:38.274101 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89069685-30d9-4814-9fb2-f645f1609410 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:38.274385 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89069685-30d9-4814-9fb2-f645f1609410 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:38.274685 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89069685-30d9-4814-9fb2-f645f1609410 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:38.275157 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b623a-0d44-48e6-a93c-0f933ffbf4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:38.275383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89069685-30d9-4814-9fb2-f645f1609410 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:38.276240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b623a-0d44-48e6-a93c-0f933ffbf4bf None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:38.276710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89069685-30d9-4814-9fb2-f645f1609410 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:38.276976 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b623a-0d44-48e6-a93c-0f933ffbf4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:38.277444 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b623a-0d44-48e6-a93c-0f933ffbf4bf None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:38.278457 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b623a-0d44-48e6-a93c-0f933ffbf4bf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:38.279006 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b623a-0d44-48e6-a93c-0f933ffbf4bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:38.280564 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b623a-0d44-48e6-a93c-0f933ffbf4bf None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:38.286511 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 447/919] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 23 11:26:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:38.290261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-89069685-30d9-4814-9fb2-f645f1609410 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:26:38.290261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:38.290261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:38.290261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:38.290261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:38.290261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:38.290261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:38.290261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:38.290261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:26:38.290261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:38.290261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:38.290261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:38.290261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:38.290261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:38.290261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:38.290261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:38.290261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:38.291503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:38.291503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:38.291503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:38.291503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:38.291503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:38.291503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:38.291503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:38.291503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:38.291503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:38.291503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:38.291503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:38.291503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:38.291503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:38.291503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:38.291503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:38.291503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:38.293710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:38.293710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:38.293710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:38.293710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:38.293710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:38.293710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:38.293710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:38.293710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:38.293710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:38.293710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:38.293710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:38.293710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:38.293710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:26:38.293710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:38.294856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b623a-0d44-48e6-a93c-0f933ffbf4bf None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:38.302602 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 473/920] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 23 11:26:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:38.302602 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89069685-30d9-4814-9fb2-f645f1609410 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:38.353656 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:38.401864 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b623a-0d44-48e6-a93c-0f933ffbf4bf None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:38.414240 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89069685-30d9-4814-9fb2-f645f1609410 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:38.437093 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71603103-4e2c-4b04-bca1-7f0ec1571ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:38.437904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71603103-4e2c-4b04-bca1-7f0ec1571ea2 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:38.438315 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71603103-4e2c-4b04-bca1-7f0ec1571ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:38.438754 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71603103-4e2c-4b04-bca1-7f0ec1571ea2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:38.439602 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71603103-4e2c-4b04-bca1-7f0ec1571ea2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:38.439717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:38.441149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71603103-4e2c-4b04-bca1-7f0ec1571ea2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:38.453637 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e59d59-8b5a-4313-885b-4db831a2abfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:38.460148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e59d59-8b5a-4313-885b-4db831a2abfe None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:38.460148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e59d59-8b5a-4313-885b-4db831a2abfe None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:38.460148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e59d59-8b5a-4313-885b-4db831a2abfe None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:38.460148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e59d59-8b5a-4313-885b-4db831a2abfe None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:38.460148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e59d59-8b5a-4313-885b-4db831a2abfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:38.460148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e59d59-8b5a-4313-885b-4db831a2abfe None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:38.462145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 474/921] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jun 23 11:26:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:38.468200 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e59d59-8b5a-4313-885b-4db831a2abfe None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:38.514330 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 448/922] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jun 23 11:26:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:26:38.578832 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e59d59-8b5a-4313-885b-4db831a2abfe None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:38.622866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d9d8f-ec4b-4c00-a5a3-c606ffb0c8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:38.623533 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d9d8f-ec4b-4c00-a5a3-c606ffb0c8dc None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:38.623814 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d9d8f-ec4b-4c00-a5a3-c606ffb0c8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:38.624086 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d9d8f-ec4b-4c00-a5a3-c606ffb0c8dc None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:38.624789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d9d8f-ec4b-4c00-a5a3-c606ffb0c8dc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:38.625092 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d9d8f-ec4b-4c00-a5a3-c606ffb0c8dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:38.626460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d9d8f-ec4b-4c00-a5a3-c606ffb0c8dc None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:38.632291 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 475/923] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jun 23 11:26:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:38.638226 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71603103-4e2c-4b04-bca1-7f0ec1571ea2 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:38.752218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71603103-4e2c-4b04-bca1-7f0ec1571ea2 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:38.768458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:38.794090 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267eb39-f1d0-4eb4-ba9a-039587626cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:38.794781 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267eb39-f1d0-4eb4-ba9a-039587626cba None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:38.795066 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267eb39-f1d0-4eb4-ba9a-039587626cba None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:38.795337 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267eb39-f1d0-4eb4-ba9a-039587626cba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:38.796133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6267eb39-f1d0-4eb4-ba9a-039587626cba None admin] RBAC: Adding query filter params (interface=public, service_id=0c8ab4d4edab4f7d990d9f4efb616476, region_id=RegionOne) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:38.796445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6267eb39-f1d0-4eb4-ba9a-039587626cba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:38.797808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6267eb39-f1d0-4eb4-ba9a-039587626cba None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:38.803367 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 449/924] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jun 23 11:26:38 2022] GET /identity/v3/endpoints?service_id=0c8ab4d4edab4f7d990d9f4efb616476&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:38.808851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d9d8f-ec4b-4c00-a5a3-c606ffb0c8dc None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:38.864858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:38.913263 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d9d8f-ec4b-4c00-a5a3-c606ffb0c8dc None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:38.947479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9c827-4749-4eda-878c-c89cabab1e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:38.948138 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9c827-4749-4eda-878c-c89cabab1e87 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:38.948422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9c827-4749-4eda-878c-c89cabab1e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:38.949224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9c827-4749-4eda-878c-c89cabab1e87 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:38.949907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9c827-4749-4eda-878c-c89cabab1e87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:38.951292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9c827-4749-4eda-878c-c89cabab1e87 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:38.957575 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 476/925] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jun 23 11:26:38 2022] GET /identity/v3/services => generated 1497 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:26:39.684612 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c3084-a667-43aa-a132-9806d29de743 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:39.685145 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c3084-a667-43aa-a132-9806d29de743 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:39.685145 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c3084-a667-43aa-a132-9806d29de743 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:39.686922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 450/926] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:39.692351 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ddf59-72f1-43b6-9ba1-7cbbca1270a8 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:39.692768 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ddf59-72f1-43b6-9ba1-7cbbca1270a8 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:39.692940 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ddf59-72f1-43b6-9ba1-7cbbca1270a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:39.710742 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-216ddf59-72f1-43b6-9ba1-7cbbca1270a8 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:39.711911 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-216ddf59-72f1-43b6-9ba1-7cbbca1270a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:39.785113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 477/927] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:26:39.810261 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c3084-a667-43aa-a132-9806d29de743 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:39.920358 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c3084-a667-43aa-a132-9806d29de743 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:39.955929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4475364-df0a-4492-b34f-900af17333d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:39.956742 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4475364-df0a-4492-b34f-900af17333d9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:39.957077 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4475364-df0a-4492-b34f-900af17333d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:39.957353 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4475364-df0a-4492-b34f-900af17333d9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:39.957997 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4475364-df0a-4492-b34f-900af17333d9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:39.959375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4475364-df0a-4492-b34f-900af17333d9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:39.972529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-b4475364-df0a-4492-b34f-900af17333d9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 11:26:39.972529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:39.972529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:39.972529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:39.972529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:39.972529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:39.972529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:39.972529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:39.972529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 11:26:39.972529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:39.972529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:39.972529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:39.972529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:39.972529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:39.972529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:39.972529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:39.973477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:39.973477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:39.973477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:39.973477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:39.973477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:39.973477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:39.973477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:39.973477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:39.973477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:39.973477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:39.973477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:39.973477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:39.973477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:39.973477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:39.973477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:39.973477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:39.974320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:39.974320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:39.974320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:39.974320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:39.974320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:39.974320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:39.974320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:39.974320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:39.974320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:39.974320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:39.974320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:39.974320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:39.974320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:39.974320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 11:26:39.974320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:39.974320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 451/928] 10.222.0.13 () {56 vars in 1132 bytes} [Thu Jun 23 11:26:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:39.979835 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ddf59-72f1-43b6-9ba1-7cbbca1270a8 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:40.022122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:40.086816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ddf59-72f1-43b6-9ba1-7cbbca1270a8 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:40.123225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ca22e-8466-4bf9-948f-02db3d0c0ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:40.123952 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ca22e-8466-4bf9-948f-02db3d0c0ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:40.124235 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ca22e-8466-4bf9-948f-02db3d0c0ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:40.124505 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ca22e-8466-4bf9-948f-02db3d0c0ff4 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:40.125218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ca22e-8466-4bf9-948f-02db3d0c0ff4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:40.125525 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ca22e-8466-4bf9-948f-02db3d0c0ff4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:40.126923 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ca22e-8466-4bf9-948f-02db3d0c0ff4 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:40.132552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 478/929] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 23 11:26:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:40.138298 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4475364-df0a-4492-b34f-900af17333d9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:40.166882 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e406c2-61ff-46a8-9294-080e17f3c9ba None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:40.167184 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e406c2-61ff-46a8-9294-080e17f3c9ba None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:40.168062 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e406c2-61ff-46a8-9294-080e17f3c9ba None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:40.169683 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 479/930] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:26:40.174625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d3c4-3dfd-44e0-b4c8-75e2118c13f0 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:40.174917 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d3c4-3dfd-44e0-b4c8-75e2118c13f0 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:40.175195 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d3c4-3dfd-44e0-b4c8-75e2118c13f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:40.183428 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:40.192907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c2f2d3c4-3dfd-44e0-b4c8-75e2118c13f0 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:40.195454 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c2f2d3c4-3dfd-44e0-b4c8-75e2118c13f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:40.249702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4475364-df0a-4492-b34f-900af17333d9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:40.278783 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 480/931] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:26:40.290707 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb23013-51db-46a8-968f-978737e2d596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:40.291372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb23013-51db-46a8-968f-978737e2d596 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:40.291651 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb23013-51db-46a8-968f-978737e2d596 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:40.291924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb23013-51db-46a8-968f-978737e2d596 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:40.293872 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb23013-51db-46a8-968f-978737e2d596 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:40.294181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb23013-51db-46a8-968f-978737e2d596 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:40.294286 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2d3c4-3dfd-44e0-b4c8-75e2118c13f0 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:40.295638 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb23013-51db-46a8-968f-978737e2d596 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:40.302763 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 452/932] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 23 11:26:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:40.308164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb23013-51db-46a8-968f-978737e2d596 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:40.339013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:40.397253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2d3c4-3dfd-44e0-b4c8-75e2118c13f0 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:40.417268 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb23013-51db-46a8-968f-978737e2d596 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:40.419774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:40.435289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8a25d-bb12-4810-a5e7-d4cd0f1e99b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:40.436110 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8a25d-bb12-4810-a5e7-d4cd0f1e99b3 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:40.436395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8a25d-bb12-4810-a5e7-d4cd0f1e99b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:40.436721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8a25d-bb12-4810-a5e7-d4cd0f1e99b3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:40.448013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8a25d-bb12-4810-a5e7-d4cd0f1e99b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:40.456218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-87e8a25d-bb12-4810-a5e7-d4cd0f1e99b3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:26:40.456218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:40.456218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:40.456218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:40.456218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:40.456218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:40.456218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:40.456218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:40.456218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:40.456218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:40.456218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:40.456218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:40.456218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:40.456218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:40.456218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:40.456218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:40.457144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:40.457144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:40.457144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:40.457144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:40.457144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:40.457144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:40.457144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:40.457144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:40.457144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:40.457144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:40.457144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:40.457144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:40.457144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:40.457144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:40.457144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:40.457144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:40.457973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:40.457973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:40.457973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:40.457973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:40.457973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:40.457973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:40.457973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:40.457973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:40.457973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:40.457973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:40.457973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:26:40.457973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:40.462324 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 481/933] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Jun 23 11:26:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:40.463403 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8a25d-bb12-4810-a5e7-d4cd0f1e99b3 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:40.465025 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dab847-0b51-40b7-b1cf-f2073ef76b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:40.465932 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dab847-0b51-40b7-b1cf-f2073ef76b6c None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:40.466412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dab847-0b51-40b7-b1cf-f2073ef76b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:40.466901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dab847-0b51-40b7-b1cf-f2073ef76b6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:40.467865 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dab847-0b51-40b7-b1cf-f2073ef76b6c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:40.469478 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dab847-0b51-40b7-b1cf-f2073ef76b6c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:40.575402 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8a25d-bb12-4810-a5e7-d4cd0f1e99b3 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:40.582885 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:40.583075 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 453/934] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jun 23 11:26:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:26:40.629556 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe31083-a6e2-45d7-91a2-5480783edb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:40.630781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe31083-a6e2-45d7-91a2-5480783edb01 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:40.631652 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe31083-a6e2-45d7-91a2-5480783edb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:40.631931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe31083-a6e2-45d7-91a2-5480783edb01 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:40.632710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe31083-a6e2-45d7-91a2-5480783edb01 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:40.632991 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe31083-a6e2-45d7-91a2-5480783edb01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:40.634392 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe31083-a6e2-45d7-91a2-5480783edb01 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:40.644621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 482/935] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jun 23 11:26:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:40.650296 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dab847-0b51-40b7-b1cf-f2073ef76b6c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:40.740965 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7cda5-6726-4a93-bf02-de5cb8928f37 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:40.741515 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7cda5-6726-4a93-bf02-de5cb8928f37 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:40.741515 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7cda5-6726-4a93-bf02-de5cb8928f37 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:40.743162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 483/936] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:40.753410 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af243f0-0557-404e-a436-0881625f76ca None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:40.753840 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af243f0-0557-404e-a436-0881625f76ca None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:40.754574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af243f0-0557-404e-a436-0881625f76ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:40.775336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dab847-0b51-40b7-b1cf-f2073ef76b6c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:40.780004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-3af243f0-0557-404e-a436-0881625f76ca None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:40.781181 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-3af243f0-0557-404e-a436-0881625f76ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:40.813575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbd409-195c-4509-aa16-564be001d95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:40.814374 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbd409-195c-4509-aa16-564be001d95f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:40.814700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbd409-195c-4509-aa16-564be001d95f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:40.814977 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbd409-195c-4509-aa16-564be001d95f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:40.826374 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bbd409-195c-4509-aa16-564be001d95f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:40.828636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bbd409-195c-4509-aa16-564be001d95f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:40.848389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-a1bbd409-195c-4509-aa16-564be001d95f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:40.848389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:40.848389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:40.848389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:40.848389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:40.848389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:40.848389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:40.848389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:40.848389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:40.848389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:40.848389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:40.848389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:40.848389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:40.848389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:40.848389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:40.849336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:40.849336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:40.849336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:40.849336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:40.849336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:40.849336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:40.849336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:40.849336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:40.849336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:40.849336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:40.849336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:40.849336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:40.849336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:40.849336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:40.849336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:40.849336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:40.849336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:40.850282 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:40.850282 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:40.850282 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:40.850282 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:40.850282 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 454/937] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:40.856554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbd409-195c-4509-aa16-564be001d95f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:40.861225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 484/938] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:26:40.871415 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:40.885088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af243f0-0557-404e-a436-0881625f76ca None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:40.961955 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbd409-195c-4509-aa16-564be001d95f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:40.982552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:41.006575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5935d69-d8b6-4175-a189-6e6608049780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:41.006575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5935d69-d8b6-4175-a189-6e6608049780 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:41.006575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5935d69-d8b6-4175-a189-6e6608049780 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:41.006575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5935d69-d8b6-4175-a189-6e6608049780 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:41.007393 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5935d69-d8b6-4175-a189-6e6608049780 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:41.007393 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5935d69-d8b6-4175-a189-6e6608049780 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:41.007703 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af243f0-0557-404e-a436-0881625f76ca None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:41.007852 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5935d69-d8b6-4175-a189-6e6608049780 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:41.019010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 455/939] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:41.020651 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ce358-35ce-4ca5-a9b0-51027a48d747 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:41.020950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ce358-35ce-4ca5-a9b0-51027a48d747 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:41.021228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ce358-35ce-4ca5-a9b0-51027a48d747 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:41.022464 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 456/940] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:41.024019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ce358-35ce-4ca5-a9b0-51027a48d747 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:41.049113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21f5ef-9793-42f3-a8d0-ee5abe2f7a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:41.049760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21f5ef-9793-42f3-a8d0-ee5abe2f7a36 None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:41.050047 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21f5ef-9793-42f3-a8d0-ee5abe2f7a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:41.050325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21f5ef-9793-42f3-a8d0-ee5abe2f7a36 None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:41.051063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21f5ef-9793-42f3-a8d0-ee5abe2f7a36 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:41.052619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21f5ef-9793-42f3-a8d0-ee5abe2f7a36 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:41.054014 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: WARNING keystone.server.flask.common [None req-fb21f5ef-9793-42f3-a8d0-ee5abe2f7a36 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 23 11:26:41.092104 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.sql.core [None req-fb21f5ef-9793-42f3-a8d0-ee5abe2f7a36 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 11:26:41.092104 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 11:26:41.092104 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [parameters: {'id': '71db0618dbcc4781be6af6e38b99e3b8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 11:26:41.092104 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80255) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 11:26:41.097255 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:41.099800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-fb21f5ef-9793-42f3-a8d0-ee5abe2f7a36 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 71db0618dbcc4781be6af6e38b99e3b8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 71db0618dbcc4781be6af6e38b99e3b8. Jun 23 11:26:41.099800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:41.099800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 11:26:41.099800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 11:26:41.099800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 11:26:41.099800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 11:26:41.099800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 11:26:41.099800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 11:26:41.099800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 11:26:41.099800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application conn.query(q) Jun 23 11:26:41.099800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 11:26:41.099800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 11:26:41.099800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 11:26:41.100658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application result.read() Jun 23 11:26:41.100658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 11:26:41.100658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 11:26:41.100658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 11:26:41.100658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 11:26:41.100658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 11:26:41.100658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 11:26:41.100658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 11:26:41.100658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 11:26:41.100658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 11:26:41.100658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:41.100658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 11:26:41.100658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:41.100658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:41.100658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 11:26:41.100658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 11:26:41.100658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 23 11:26:41.100658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 23 11:26:41.101630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 11:26:41.101630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application next(self.gen) Jun 23 11:26:41.101630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 11:26:41.101630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application yield resource Jun 23 11:26:41.101630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 11:26:41.101630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application next(self.gen) Jun 23 11:26:41.101630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 11:26:41.101630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 11:26:41.101630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 11:26:41.101630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 11:26:41.101630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 11:26:41.101630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise self.value Jun 23 11:26:41.101630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 11:26:41.101630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 11:26:41.101630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 11:26:41.101630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application session.commit() Jun 23 11:26:41.101630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 11:26:41.102546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 11:26:41.102546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 11:26:41.102546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 11:26:41.102546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 11:26:41.102546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self.session.flush() Jun 23 11:26:41.102546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 11:26:41.102546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 11:26:41.102546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 11:26:41.102546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 11:26:41.102546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 11:26:41.102546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application compat.raise_( Jun 23 11:26:41.102546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 11:26:41.102546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception Jun 23 11:26:41.102546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 11:26:41.102546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 11:26:41.102546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 11:26:41.102546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 11:26:41.104248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 11:26:41.104248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 11:26:41.104248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 11:26:41.104248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 11:26:41.104248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 11:26:41.104248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 11:26:41.104248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 11:26:41.104248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 11:26:41.104248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 11:26:41.104248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 11:26:41.104248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 11:26:41.104248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 11:26:41.104248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 11:26:41.104248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 11:26:41.104248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 11:26:41.105217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 11:26:41.105217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 11:26:41.105217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception Jun 23 11:26:41.105217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 11:26:41.105217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 11:26:41.105217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 11:26:41.105217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 11:26:41.105217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 11:26:41.105217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 11:26:41.105217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 11:26:41.105217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application conn.query(q) Jun 23 11:26:41.105217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 11:26:41.105217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 11:26:41.105217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 11:26:41.105217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application result.read() Jun 23 11:26:41.105217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 11:26:41.105217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 11:26:41.109198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 11:26:41.109198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 11:26:41.109198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 11:26:41.109198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 11:26:41.109198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 11:26:41.109198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 11:26:41.109198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 11:26:41.109198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 11:26:41.109198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [parameters: {'id': '71db0618dbcc4781be6af6e38b99e3b8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 11:26:41.109198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 11:26:41.109198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:41.109198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:41.109198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:41.109198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:41.109198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 23 11:26:41.110016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 23 11:26:41.110016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 11:26:41.110016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 11:26:41.110016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 23 11:26:41.110016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:41.110016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:41.110016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:41.110016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:41.110016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:41.110016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:41.110016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:41.110016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:41.110016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:41.110016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:41.110016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:41.110016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:41.110016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:41.111004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:41.111004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 23 11:26:41.111004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 23 11:26:41.111004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:41.111004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:41.111004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 23 11:26:41.111004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 23 11:26:41.111004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 71db0618dbcc4781be6af6e38b99e3b8. Jun 23 11:26:41.111004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:41.111004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 485/941] 10.222.0.13 () {60 vars in 1140 bytes} [Thu Jun 23 11:26:40 2022] POST /identity/v3/projects => generated 262 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:41.111004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e579666-cf7c-4def-9508-23871a6eb813 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:41.111004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e579666-cf7c-4def-9508-23871a6eb813 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:41.111810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e579666-cf7c-4def-9508-23871a6eb813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:41.126232 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-5e579666-cf7c-4def-9508-23871a6eb813 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:41.127865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-5e579666-cf7c-4def-9508-23871a6eb813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:41.136963 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ce358-35ce-4ca5-a9b0-51027a48d747 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:41.172368 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da97f01-d06a-46a7-be32-9e873bf60fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:41.173183 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da97f01-d06a-46a7-be32-9e873bf60fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:41.173468 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da97f01-d06a-46a7-be32-9e873bf60fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:41.173744 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da97f01-d06a-46a7-be32-9e873bf60fb7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:41.182897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da97f01-d06a-46a7-be32-9e873bf60fb7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:41.184631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da97f01-d06a-46a7-be32-9e873bf60fb7 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:41.200570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-6da97f01-d06a-46a7-be32-9e873bf60fb7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:26:41.200570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:41.200570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:41.200570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:41.200570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:41.200570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:41.200570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:41.200570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:41.200570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:41.200570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:41.200570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:41.200570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:41.200570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:41.200570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:41.200570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:41.200570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:41.201565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:41.201565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:41.201565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:41.201565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:41.201565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:41.201565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:41.201565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:41.201565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:41.201565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:41.201565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:41.201565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:41.201565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:41.201565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:41.201565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:41.201565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:41.201565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:41.201565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:41.201565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:41.206856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:41.206856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:41.206856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:41.206856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:41.206856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:41.206856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:41.206856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:41.206856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:41.206856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:26:41.206856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:41.207437 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 486/942] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:41.207437 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e579666-cf7c-4def-9508-23871a6eb813 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:41.207584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 457/943] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jun 23 11:26:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:41.207682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da97f01-d06a-46a7-be32-9e873bf60fb7 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:41.280651 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:41.316304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e579666-cf7c-4def-9508-23871a6eb813 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:41.328230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da97f01-d06a-46a7-be32-9e873bf60fb7 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:41.351493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7fa347-1786-471d-a603-287b4d140dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:41.352495 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fa347-1786-471d-a603-287b4d140dcf None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:41.352856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fa347-1786-471d-a603-287b4d140dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:41.353199 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fa347-1786-471d-a603-287b4d140dcf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:41.359276 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:41.360691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7fa347-1786-471d-a603-287b4d140dcf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:41.362349 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7fa347-1786-471d-a603-287b4d140dcf None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:41.365880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d10d90-083f-453b-afcf-ae1e790846fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:41.366623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d10d90-083f-453b-afcf-ae1e790846fc None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:41.366895 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d10d90-083f-453b-afcf-ae1e790846fc None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:41.368938 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d10d90-083f-453b-afcf-ae1e790846fc None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:41.368938 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d10d90-083f-453b-afcf-ae1e790846fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:41.368938 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d10d90-083f-453b-afcf-ae1e790846fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:41.369751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d10d90-083f-453b-afcf-ae1e790846fc None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:41.376918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-5d7fa347-1786-471d-a603-287b4d140dcf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:41.376918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:41.376918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:41.376918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:41.376918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:41.376918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:41.376918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:41.376918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:41.376918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:41.376918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:41.376918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:41.376918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:41.376918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:41.376918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:41.376918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:41.377827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:41.377827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:41.377827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:41.377827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:41.377827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:41.377827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:41.377827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:41.377827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:41.377827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:41.377827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:41.377827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:41.377827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:41.377827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:41.377827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:41.377827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:41.377827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:41.377827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:41.378816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:41.378816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:41.378816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:41.378816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:41.378816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:41.378816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:41.378816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:41.378816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:41.378816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 487/944] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:41.379991 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7fa347-1786-471d-a603-287b4d140dcf None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:41.398859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 458/945] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Jun 23 11:26:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:41.399825 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d10d90-083f-453b-afcf-ae1e790846fc None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:41.490062 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7fa347-1786-471d-a603-287b4d140dcf None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:41.518558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:41.520828 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d10d90-083f-453b-afcf-ae1e790846fc None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:41.527243 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4fd277-3529-44a8-a7f0-ec14b4976545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:41.536212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fd277-3529-44a8-a7f0-ec14b4976545 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:41.536212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fd277-3529-44a8-a7f0-ec14b4976545 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:41.536212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fd277-3529-44a8-a7f0-ec14b4976545 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:41.536212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4fd277-3529-44a8-a7f0-ec14b4976545 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:41.536212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4fd277-3529-44a8-a7f0-ec14b4976545 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:41.544468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-ec4fd277-3529-44a8-a7f0-ec14b4976545 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:26:41.544468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:41.544468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:41.544468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:41.544468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:41.544468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:41.544468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:41.544468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:41.544468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:26:41.544468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:41.544468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:41.544468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:41.544468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:41.544468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:41.544468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:41.544468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:41.544468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:41.545656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:41.545656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:41.545656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:41.545656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:41.545656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:41.545656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:41.545656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:41.545656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:41.545656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:41.545656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:41.545656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:41.545656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:41.545656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:41.545656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:41.545656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:41.545656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:41.546708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:41.546708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:41.546708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:41.546708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:41.546708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:41.546708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:41.546708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:41.546708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:41.546708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:41.546708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:41.546708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:41.546708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:41.546708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:26:41.546708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:41.550653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 488/946] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 23 11:26:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:41.551476 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4fd277-3529-44a8-a7f0-ec14b4976545 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:41.561524 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0fb75-ca28-413a-ad3e-4e13083f75b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:41.562638 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0fb75-ca28-413a-ad3e-4e13083f75b5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:41.562933 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0fb75-ca28-413a-ad3e-4e13083f75b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:41.563214 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0fb75-ca28-413a-ad3e-4e13083f75b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:41.564332 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0fb75-ca28-413a-ad3e-4e13083f75b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:41.564651 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0fb75-ca28-413a-ad3e-4e13083f75b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:41.566422 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0fb75-ca28-413a-ad3e-4e13083f75b5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:41.594564 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 459/947] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 23 11:26:41 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:41.594564 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0fb75-ca28-413a-ad3e-4e13083f75b5 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:41.644490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:41.690097 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4fd277-3529-44a8-a7f0-ec14b4976545 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:41.704385 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0fb75-ca28-413a-ad3e-4e13083f75b5 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:41.730765 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12156a-2da7-4159-a7c7-5a821f7557f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:41.731760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12156a-2da7-4159-a7c7-5a821f7557f1 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:41.732048 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12156a-2da7-4159-a7c7-5a821f7557f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:41.732328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12156a-2da7-4159-a7c7-5a821f7557f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:41.743333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c264a-9891-440b-a3e9-79e6468c981b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:41.744905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c264a-9891-440b-a3e9-79e6468c981b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:41.744905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c264a-9891-440b-a3e9-79e6468c981b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:41.744905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c264a-9891-440b-a3e9-79e6468c981b None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:41.745236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12156a-2da7-4159-a7c7-5a821f7557f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:41.746290 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c264a-9891-440b-a3e9-79e6468c981b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:41.746290 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c264a-9891-440b-a3e9-79e6468c981b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:41.749843 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12156a-2da7-4159-a7c7-5a821f7557f1 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:41.750065 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c264a-9891-440b-a3e9-79e6468c981b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:41.754473 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 460/948] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jun 23 11:26:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:41.760568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c264a-9891-440b-a3e9-79e6468c981b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:41.767237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-cb12156a-2da7-4159-a7c7-5a821f7557f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:41.767237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:41.767237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:41.767237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:41.767237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:41.767237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:41.767237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:41.767237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:41.767237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:41.767237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:41.767237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:41.767237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:41.767237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:41.767237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:41.767237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:41.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:41.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:41.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:41.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:41.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:41.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:41.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:41.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:41.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:41.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:41.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:41.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:41.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:41.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:41.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:41.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:41.775746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:41.776669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:41.776669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:41.776669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:41.776669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:41.776669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 489/949] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:41.776669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12156a-2da7-4159-a7c7-5a821f7557f1 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:41.837252 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:41.891716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c264a-9891-440b-a3e9-79e6468c981b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:41.905229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12156a-2da7-4159-a7c7-5a821f7557f1 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:41.909855 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:41.935309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d109d-a311-45e2-a578-0df7af9e894f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:41.936496 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d109d-a311-45e2-a578-0df7af9e894f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:41.936956 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d109d-a311-45e2-a578-0df7af9e894f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:41.937527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d109d-a311-45e2-a578-0df7af9e894f None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:41.938815 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d109d-a311-45e2-a578-0df7af9e894f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:41.939543 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d109d-a311-45e2-a578-0df7af9e894f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:41.941278 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d109d-a311-45e2-a578-0df7af9e894f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:41.947413 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ff9e9-0c4f-4b7f-8c37-0f28d0361101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:41.952922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ff9e9-0c4f-4b7f-8c37-0f28d0361101 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:41.952922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ff9e9-0c4f-4b7f-8c37-0f28d0361101 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:41.952922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ff9e9-0c4f-4b7f-8c37-0f28d0361101 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:41.952922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ff9e9-0c4f-4b7f-8c37-0f28d0361101 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:41.952922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ff9e9-0c4f-4b7f-8c37-0f28d0361101 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:41.952922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ff9e9-0c4f-4b7f-8c37-0f28d0361101 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:41.953662 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 461/950] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jun 23 11:26:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:41.955498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d109d-a311-45e2-a578-0df7af9e894f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:41.970579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 490/951] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:41.975825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ff9e9-0c4f-4b7f-8c37-0f28d0361101 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:42.080031 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d109d-a311-45e2-a578-0df7af9e894f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:42.088753 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ff9e9-0c4f-4b7f-8c37-0f28d0361101 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:42.092367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:42.119483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755bd0c4-397e-4a17-9ffe-c62e8b575140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:42.121181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bd0c4-397e-4a17-9ffe-c62e8b575140 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:42.121687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bd0c4-397e-4a17-9ffe-c62e8b575140 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:42.122073 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bd0c4-397e-4a17-9ffe-c62e8b575140 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:42.122982 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755bd0c4-397e-4a17-9ffe-c62e8b575140 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:42.124466 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755bd0c4-397e-4a17-9ffe-c62e8b575140 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:42.126026 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816e6cb-da3d-4eb0-97fe-0bc3bbb9e0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:42.127018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816e6cb-da3d-4eb0-97fe-0bc3bbb9e0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:42.127449 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816e6cb-da3d-4eb0-97fe-0bc3bbb9e0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:42.127868 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816e6cb-da3d-4eb0-97fe-0bc3bbb9e0f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:42.134966 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816e6cb-da3d-4eb0-97fe-0bc3bbb9e0f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:42.136569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816e6cb-da3d-4eb0-97fe-0bc3bbb9e0f5 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:42.150596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-9816e6cb-da3d-4eb0-97fe-0bc3bbb9e0f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:42.150596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:42.150596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:42.150596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:42.150596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:42.150596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:42.150596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:42.150596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:42.150596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:42.150596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:42.150596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:42.150596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:42.150596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:42.150596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:42.150596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:42.151558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:42.151558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:42.151558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:42.151558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:42.151558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:42.151558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:42.151558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:42.151558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:42.151558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:42.151558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:42.151558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:42.151558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:42.151558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:42.151558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:42.151558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:42.151558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:42.151558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:42.153041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:42.153041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:42.153041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:42.153041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:42.153041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:42.153041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:42.153041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:42.153041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:42.153041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 491/952] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:42.157824 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816e6cb-da3d-4eb0-97fe-0bc3bbb9e0f5 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:42.207210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 462/953] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jun 23 11:26:41 2022] POST /identity/v3/endpoints => generated 349 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:26:42.281654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816e6cb-da3d-4eb0-97fe-0bc3bbb9e0f5 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:42.328796 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e5552-55b4-431f-b22c-4faf5d6579b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:42.329911 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e5552-55b4-431f-b22c-4faf5d6579b6 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:42.330410 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e5552-55b4-431f-b22c-4faf5d6579b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:42.330927 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e5552-55b4-431f-b22c-4faf5d6579b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:42.331955 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e5552-55b4-431f-b22c-4faf5d6579b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:42.332592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e5552-55b4-431f-b22c-4faf5d6579b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:42.334281 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e5552-55b4-431f-b22c-4faf5d6579b6 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:42.349854 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 492/954] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:42.356764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755bd0c4-397e-4a17-9ffe-c62e8b575140 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:42.391000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:42.465246 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755bd0c4-397e-4a17-9ffe-c62e8b575140 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:42.513472 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd60e4-698b-464f-9c88-85397e943c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:42.514374 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd60e4-698b-464f-9c88-85397e943c7a None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:42.514697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd60e4-698b-464f-9c88-85397e943c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:42.514973 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd60e4-698b-464f-9c88-85397e943c7a None admin] PATH_INFO: `/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/030614ef56f143f8be6fcc0445ec9568/roles/a9a87bede27348b89fa09b9dfd04e89c` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:42.517219 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd60e4-698b-464f-9c88-85397e943c7a None admin] RBAC: Authorizing `identity:create_grant(project_id=886c2939a14b48febc2e2fe639c8f1e4, user_id=030614ef56f143f8be6fcc0445ec9568, role_id=a9a87bede27348b89fa09b9dfd04e89c)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:42.518700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd60e4-698b-464f-9c88-85397e943c7a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:42.584979 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 463/955] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:26:42 2022] PUT /identity/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/030614ef56f143f8be6fcc0445ec9568/roles/a9a87bede27348b89fa09b9dfd04e89c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:26:42.739676 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d8b99-6528-4746-aa1e-1e5898420a5a None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:42.740095 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d8b99-6528-4746-aa1e-1e5898420a5a None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:42.740371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d8b99-6528-4746-aa1e-1e5898420a5a None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:42.741873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 493/956] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:42.747963 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4b8a1-9d99-4646-b1e1-bffdcdeea9e4 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:42.748502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4b8a1-9d99-4646-b1e1-bffdcdeea9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:42.748972 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4b8a1-9d99-4646-b1e1-bffdcdeea9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:42.767156 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-9ba4b8a1-9d99-4646-b1e1-bffdcdeea9e4 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:42.768326 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9ba4b8a1-9d99-4646-b1e1-bffdcdeea9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:42.841086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 464/957] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:26:42.863400 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d8b99-6528-4746-aa1e-1e5898420a5a None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:42.887328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:42.975055 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d8b99-6528-4746-aa1e-1e5898420a5a None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:43.012586 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f27de1-90db-42c1-a4e5-9d097f02c6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:43.013702 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f27de1-90db-42c1-a4e5-9d097f02c6ff None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:43.014184 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f27de1-90db-42c1-a4e5-9d097f02c6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:43.014681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f27de1-90db-42c1-a4e5-9d097f02c6ff None admin] PATH_INFO: `/v3/services/compute` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:43.015565 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f27de1-90db-42c1-a4e5-9d097f02c6ff None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:43.017126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f27de1-90db-42c1-a4e5-9d097f02c6ff None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:43.030112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-67f27de1-90db-42c1-a4e5-9d097f02c6ff None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 11:26:43.030112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:43.030112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:43.030112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:43.030112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:43.030112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:43.030112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:43.030112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:43.030112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 11:26:43.030112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:43.030112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:43.030112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:43.030112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:43.030112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:43.030112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:43.030112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:43.030112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:43.031317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:43.031317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:43.031317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:43.031317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:43.031317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:43.031317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:43.031317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:43.031317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:43.031317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:43.031317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:43.031317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:43.031317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:43.031317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:43.031317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:43.031317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:43.031317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:43.032185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:43.032185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:43.032185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:43.032185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:43.032185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:43.032185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:43.032185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:43.032185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:43.032185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:43.032185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:43.032185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:43.032185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:43.032185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 11:26:43.032185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:43.035035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 494/958] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:43.043786 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4b8a1-9d99-4646-b1e1-bffdcdeea9e4 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:43.066606 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:43.155497 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4b8a1-9d99-4646-b1e1-bffdcdeea9e4 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:43.192631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e221e-237a-49e6-af35-3b9a6dfb28e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:43.193661 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e221e-237a-49e6-af35-3b9a6dfb28e6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:43.194142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e221e-237a-49e6-af35-3b9a6dfb28e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:43.194642 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e221e-237a-49e6-af35-3b9a6dfb28e6 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:43.195537 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e221e-237a-49e6-af35-3b9a6dfb28e6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:43.196035 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e221e-237a-49e6-af35-3b9a6dfb28e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:43.197615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e221e-237a-49e6-af35-3b9a6dfb28e6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:43.204125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 465/959] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 23 11:26:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:43.211611 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f27de1-90db-42c1-a4e5-9d097f02c6ff None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:43.298689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:43.315698 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f27de1-90db-42c1-a4e5-9d097f02c6ff None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:43.351891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356a981-7a69-425f-a6b9-e59a3388fb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:43.352978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356a981-7a69-425f-a6b9-e59a3388fb70 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:43.353469 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356a981-7a69-425f-a6b9-e59a3388fb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:43.353942 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356a981-7a69-425f-a6b9-e59a3388fb70 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:43.355517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356a981-7a69-425f-a6b9-e59a3388fb70 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:43.356624 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356a981-7a69-425f-a6b9-e59a3388fb70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:43.358202 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356a981-7a69-425f-a6b9-e59a3388fb70 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:43.363996 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 495/960] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 23 11:26:43 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:43.830850 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878a539-00ac-436e-a430-f91e85f196d5 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:43.831282 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878a539-00ac-436e-a430-f91e85f196d5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:43.831469 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878a539-00ac-436e-a430-f91e85f196d5 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:43.833335 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 466/961] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:43.838978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890240c-772b-4a57-aa58-03d4dfb29428 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:43.839833 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890240c-772b-4a57-aa58-03d4dfb29428 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:43.839833 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890240c-772b-4a57-aa58-03d4dfb29428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:43.856383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-d890240c-772b-4a57-aa58-03d4dfb29428 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:43.857546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d890240c-772b-4a57-aa58-03d4dfb29428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:43.927862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 496/962] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:43.953296 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878a539-00ac-436e-a430-f91e85f196d5 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:44.044746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:44.062308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878a539-00ac-436e-a430-f91e85f196d5 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:44.098750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f77ad-2dd6-4506-b0a1-38ca05dc4818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:44.099342 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f77ad-2dd6-4506-b0a1-38ca05dc4818 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:44.099627 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f77ad-2dd6-4506-b0a1-38ca05dc4818 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:44.099903 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f77ad-2dd6-4506-b0a1-38ca05dc4818 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:44.100621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f77ad-2dd6-4506-b0a1-38ca05dc4818 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:44.102059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f77ad-2dd6-4506-b0a1-38ca05dc4818 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:44.103392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: WARNING keystone.server.flask.common [None req-430f77ad-2dd6-4506-b0a1-38ca05dc4818 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 23 11:26:44.145871 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.sql.core [None req-430f77ad-2dd6-4506-b0a1-38ca05dc4818 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 11:26:44.145871 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 11:26:44.145871 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [parameters: {'user_id': 'e5f93388aa9941beb18908bf9a2233e9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 11:26:44.145871 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80256) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 11:26:44.163039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-430f77ad-2dd6-4506-b0a1-38ca05dc4818 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 11:26:44.163039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:44.163039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 11:26:44.163039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 11:26:44.163039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 11:26:44.163039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 11:26:44.163039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 11:26:44.163039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 11:26:44.163039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 11:26:44.163039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application conn.query(q) Jun 23 11:26:44.163039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 11:26:44.163039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 11:26:44.163039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 11:26:44.163039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application result.read() Jun 23 11:26:44.163039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 11:26:44.163935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 11:26:44.163935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 11:26:44.163935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 11:26:44.163935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 11:26:44.163935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 11:26:44.163935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 11:26:44.163935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 11:26:44.163935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 11:26:44.163935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:44.163935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 11:26:44.163935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:44.163935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:44.163935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 11:26:44.163935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 11:26:44.163935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 23 11:26:44.163935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 23 11:26:44.163935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 11:26:44.163935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application next(self.gen) Jun 23 11:26:44.164911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 11:26:44.164911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application yield resource Jun 23 11:26:44.164911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 11:26:44.164911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application next(self.gen) Jun 23 11:26:44.164911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 11:26:44.164911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 11:26:44.164911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 11:26:44.164911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 11:26:44.164911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 11:26:44.164911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise self.value Jun 23 11:26:44.164911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 11:26:44.164911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 11:26:44.164911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 11:26:44.164911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application session.commit() Jun 23 11:26:44.164911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 11:26:44.164911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 11:26:44.164911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 11:26:44.165813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 11:26:44.165813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 11:26:44.165813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self.session.flush() Jun 23 11:26:44.165813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 11:26:44.165813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 11:26:44.165813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 11:26:44.165813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 11:26:44.165813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 11:26:44.165813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application compat.raise_( Jun 23 11:26:44.165813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 11:26:44.165813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception Jun 23 11:26:44.165813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 11:26:44.165813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 11:26:44.165813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 11:26:44.165813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 11:26:44.165813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 11:26:44.165813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 11:26:44.166740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 11:26:44.166740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 11:26:44.166740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 23 11:26:44.166740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 23 11:26:44.166740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 11:26:44.166740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 11:26:44.166740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 11:26:44.166740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 11:26:44.166740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 11:26:44.166740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 11:26:44.166740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 11:26:44.166740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 11:26:44.166740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 11:26:44.166740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 11:26:44.166740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 11:26:44.166740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception Jun 23 11:26:44.169953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 11:26:44.169953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 11:26:44.169953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 11:26:44.169953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 11:26:44.169953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 11:26:44.169953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 11:26:44.169953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 11:26:44.169953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application conn.query(q) Jun 23 11:26:44.169953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 11:26:44.169953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 11:26:44.169953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 11:26:44.169953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application result.read() Jun 23 11:26:44.169953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 11:26:44.169953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 11:26:44.169953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 11:26:44.169953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 11:26:44.171966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 11:26:44.171966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 11:26:44.171966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 11:26:44.171966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 11:26:44.171966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 11:26:44.171966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 11:26:44.171966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e5f93388aa9941beb18908bf9a2233e9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 11:26:44.171966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 11:26:44.171966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:44.171966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:44.171966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:44.171966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:44.171966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:44.171966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:44.171966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:44.173848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:44.173848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:44.173848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:44.173848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:44.173848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:44.173848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:44.173848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:44.173848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 23 11:26:44.173848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 23 11:26:44.173848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:44.173848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:44.173848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:44.173848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:44.173848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:44.173848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:44.173848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 23 11:26:44.173848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 23 11:26:44.176244 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 23 11:26:44.176244 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 23 11:26:44.176244 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 11:26:44.176244 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 11:26:44.176244 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 11:26:44.176244 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:44.176244 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 467/963] 10.222.0.13 () {60 vars in 1131 bytes} [Thu Jun 23 11:26:43 2022] POST /identity/v3/users => generated 160 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:44.177131 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890240c-772b-4a57-aa58-03d4dfb29428 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:44.256540 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:44.284775 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890240c-772b-4a57-aa58-03d4dfb29428 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:44.319502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95edb1-d95e-4731-916a-651a7178a6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:44.320578 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95edb1-d95e-4731-916a-651a7178a6de None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:44.321011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95edb1-d95e-4731-916a-651a7178a6de None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:44.321447 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95edb1-d95e-4731-916a-651a7178a6de None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:44.328985 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95edb1-d95e-4731-916a-651a7178a6de None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:44.330606 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95edb1-d95e-4731-916a-651a7178a6de None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:44.345924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-8f95edb1-d95e-4731-916a-651a7178a6de None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 11:26:44.345924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:44.345924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:44.345924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:44.345924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:44.345924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:44.345924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:44.345924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:44.345924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:44.345924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:44.345924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:44.345924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:44.345924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:44.345924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:44.345924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:44.345924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:44.347141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:44.347141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:44.347141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:44.347141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:44.347141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:44.347141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:44.347141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:44.347141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:44.347141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:44.347141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:44.347141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:44.347141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:44.347141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:44.347141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:44.347141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:44.347141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:44.347141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:44.347141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:44.348382 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:44.348382 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:44.348382 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:44.348382 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:44.348382 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:44.348382 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:44.348382 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:44.348382 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:44.348382 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 11:26:44.348382 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:44.350755 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 497/964] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:26:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:44.357321 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d75813-8d36-4a88-8c4f-21e768907261 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:44.357601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d75813-8d36-4a88-8c4f-21e768907261 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:44.357886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d75813-8d36-4a88-8c4f-21e768907261 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:44.358922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95edb1-d95e-4731-916a-651a7178a6de None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:44.362736 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 468/965] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:44.367483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1e31d-d330-4ec4-816a-c8fb5b7b31bd None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:44.367767 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1e31d-d330-4ec4-816a-c8fb5b7b31bd None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:44.368044 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1e31d-d330-4ec4-816a-c8fb5b7b31bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:44.392445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-84b1e31d-d330-4ec4-816a-c8fb5b7b31bd None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:44.394311 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-84b1e31d-d330-4ec4-816a-c8fb5b7b31bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:44.478135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95edb1-d95e-4731-916a-651a7178a6de None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:44.481106 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 469/966] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:44.508872 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1e31d-d330-4ec4-816a-c8fb5b7b31bd None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:44.525859 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c7689-166a-4c82-a3c7-6afb6a371fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:44.527980 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c7689-166a-4c82-a3c7-6afb6a371fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:44.528523 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c7689-166a-4c82-a3c7-6afb6a371fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:44.528805 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c7689-166a-4c82-a3c7-6afb6a371fb7 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:44.530000 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c7689-166a-4c82-a3c7-6afb6a371fb7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:44.530345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c7689-166a-4c82-a3c7-6afb6a371fb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:44.531813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c7689-166a-4c82-a3c7-6afb6a371fb7 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:44.556035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:44.584448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 498/967] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 23 11:26:44 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:44.606566 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:44.632378 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1e31d-d330-4ec4-816a-c8fb5b7b31bd None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:44.674268 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691d3fc-3360-40dc-a59a-c949b839a644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:44.675533 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691d3fc-3360-40dc-a59a-c949b839a644 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:44.676039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691d3fc-3360-40dc-a59a-c949b839a644 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:44.676609 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691d3fc-3360-40dc-a59a-c949b839a644 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:44.677500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691d3fc-3360-40dc-a59a-c949b839a644 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:44.679282 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691d3fc-3360-40dc-a59a-c949b839a644 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:44.693569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-1691d3fc-3360-40dc-a59a-c949b839a644 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:26:44.693569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:44.693569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:44.693569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:44.693569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:44.693569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:44.693569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:44.693569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:44.693569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:26:44.693569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:44.693569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:44.693569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:44.693569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:44.693569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:44.693569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:44.693569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:44.693569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:44.694688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:44.694688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:44.694688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:44.694688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:44.694688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:44.694688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:44.694688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:44.694688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:44.694688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:44.694688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:44.694688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:44.694688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:44.694688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:44.694688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:44.694688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:44.694688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:44.695529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:44.695529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:44.695529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:44.695529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:44.695529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:44.695529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:44.695529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:44.695529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:44.695529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:44.695529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:44.695529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:44.695529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:44.695529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:26:44.695529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:44.699036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 470/968] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 23 11:26:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:44.705091 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c7689-166a-4c82-a3c7-6afb6a371fb7 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:44.807329 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abab66-942b-4eb2-b55a-f930cc6d936b None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:44.807863 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abab66-942b-4eb2-b55a-f930cc6d936b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:44.808281 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abab66-942b-4eb2-b55a-f930cc6d936b None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:44.809998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 471/969] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:26:44.815911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba062675-a0c1-4e6d-9466-2281c40779e3 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:44.816396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba062675-a0c1-4e6d-9466-2281c40779e3 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:44.816661 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c7689-166a-4c82-a3c7-6afb6a371fb7 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:44.816767 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba062675-a0c1-4e6d-9466-2281c40779e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:44.836489 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ba062675-a0c1-4e6d-9466-2281c40779e3 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:44.837674 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ba062675-a0c1-4e6d-9466-2281c40779e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:44.858852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5debc9e6-c7ed-486d-bcdb-607666c0c5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:44.859474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debc9e6-c7ed-486d-bcdb-607666c0c5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:44.859734 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debc9e6-c7ed-486d-bcdb-607666c0c5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:44.860016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debc9e6-c7ed-486d-bcdb-607666c0c5e6 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:44.862403 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5debc9e6-c7ed-486d-bcdb-607666c0c5e6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:44.862746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5debc9e6-c7ed-486d-bcdb-607666c0c5e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:44.864751 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5debc9e6-c7ed-486d-bcdb-607666c0c5e6 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:44.875335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 499/970] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jun 23 11:26:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:44.883673 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5debc9e6-c7ed-486d-bcdb-607666c0c5e6 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:44.921282 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 472/971] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:26:44.945554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba062675-a0c1-4e6d-9466-2281c40779e3 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:44.952318 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:44.998937 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5debc9e6-c7ed-486d-bcdb-607666c0c5e6 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:45.019486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:45.035699 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdc613-dd5f-4618-88c6-d6b8668f2bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:45.036367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdc613-dd5f-4618-88c6-d6b8668f2bbf None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:45.036978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdc613-dd5f-4618-88c6-d6b8668f2bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:45.036978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdc613-dd5f-4618-88c6-d6b8668f2bbf None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:45.037616 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cdc613-dd5f-4618-88c6-d6b8668f2bbf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:45.037949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cdc613-dd5f-4618-88c6-d6b8668f2bbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:45.039330 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cdc613-dd5f-4618-88c6-d6b8668f2bbf None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:45.049562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 500/972] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jun 23 11:26:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:45.050570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdc613-dd5f-4618-88c6-d6b8668f2bbf None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:45.054597 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba062675-a0c1-4e6d-9466-2281c40779e3 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:45.090631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058de81-cb84-4576-afcc-a07d5fc94e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:45.091448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058de81-cb84-4576-afcc-a07d5fc94e2c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:45.091733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058de81-cb84-4576-afcc-a07d5fc94e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:45.092007 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058de81-cb84-4576-afcc-a07d5fc94e2c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:45.102177 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d058de81-cb84-4576-afcc-a07d5fc94e2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:45.110634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-d058de81-cb84-4576-afcc-a07d5fc94e2c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:26:45.110634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:45.110634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:45.110634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:45.110634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:45.110634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:45.110634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:45.110634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:45.110634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:45.110634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:45.110634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:45.110634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:45.110634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:45.110634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:45.110634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:45.110634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:45.111559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:45.111559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:45.111559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:45.111559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:45.111559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:45.111559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:45.111559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:45.111559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:45.111559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:45.111559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:45.111559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:45.111559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:45.111559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:45.111559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:45.111559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:45.111559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:45.112404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:45.112404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:45.112404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:45.112404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:45.112404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:45.112404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:45.112404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:45.112404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:45.112404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:45.112404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:45.112404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:26:45.112404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:45.116619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 473/973] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Jun 23 11:26:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:45.117365 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058de81-cb84-4576-afcc-a07d5fc94e2c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:45.157919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdc613-dd5f-4618-88c6-d6b8668f2bbf None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:45.182376 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:45.197453 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8034e2-d497-44d5-8098-c3f994dd3ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:45.198117 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8034e2-d497-44d5-8098-c3f994dd3ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:45.198428 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8034e2-d497-44d5-8098-c3f994dd3ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:45.198732 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8034e2-d497-44d5-8098-c3f994dd3ec3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:45.199569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8034e2-d497-44d5-8098-c3f994dd3ec3 None admin] RBAC: Adding query filter params (interface=admin, service_id=0c8ab4d4edab4f7d990d9f4efb616476, region_id=RegionOne) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:45.199881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8034e2-d497-44d5-8098-c3f994dd3ec3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:45.201249 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8034e2-d497-44d5-8098-c3f994dd3ec3 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:45.213038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 501/974] 10.222.0.13 () {56 vars in 1249 bytes} [Thu Jun 23 11:26:45 2022] GET /identity/v3/endpoints?service_id=0c8ab4d4edab4f7d990d9f4efb616476&interface=admin®ion_id=RegionOne => generated 193 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:45.215038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8034e2-d497-44d5-8098-c3f994dd3ec3 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:45.232493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058de81-cb84-4576-afcc-a07d5fc94e2c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:45.271288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:45.277178 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84c60f-5504-4490-b095-35f483f63d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:45.277929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84c60f-5504-4490-b095-35f483f63d5c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:45.278215 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84c60f-5504-4490-b095-35f483f63d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:45.278668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84c60f-5504-4490-b095-35f483f63d5c None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:45.279259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84c60f-5504-4490-b095-35f483f63d5c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:45.279565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84c60f-5504-4490-b095-35f483f63d5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:45.280936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84c60f-5504-4490-b095-35f483f63d5c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:45.289917 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 474/975] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jun 23 11:26:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:45.295886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84c60f-5504-4490-b095-35f483f63d5c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:45.319015 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8034e2-d497-44d5-8098-c3f994dd3ec3 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:45.357354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909b237-401d-48fd-bed8-64967ef8c669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:45.358334 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909b237-401d-48fd-bed8-64967ef8c669 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:45.358903 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909b237-401d-48fd-bed8-64967ef8c669 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:45.359185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909b237-401d-48fd-bed8-64967ef8c669 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:45.360013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909b237-401d-48fd-bed8-64967ef8c669 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:45.362046 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909b237-401d-48fd-bed8-64967ef8c669 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:45.370984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 502/976] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jun 23 11:26:45 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:26:45.414371 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84c60f-5504-4490-b095-35f483f63d5c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:45.454980 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898ac99-46ae-4f4e-8ff3-a5f745007014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:45.455800 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898ac99-46ae-4f4e-8ff3-a5f745007014 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:45.456086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898ac99-46ae-4f4e-8ff3-a5f745007014 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:45.456361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898ac99-46ae-4f4e-8ff3-a5f745007014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:45.468340 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898ac99-46ae-4f4e-8ff3-a5f745007014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:45.469751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898ac99-46ae-4f4e-8ff3-a5f745007014 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:45.491239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-5898ac99-46ae-4f4e-8ff3-a5f745007014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:45.491239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:45.491239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:45.491239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:45.491239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:45.491239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:45.491239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:45.491239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:45.491239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:45.491239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:45.491239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:45.491239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:45.491239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:45.491239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:45.491239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:45.492221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:45.492221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:45.492221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:45.492221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:45.492221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:45.492221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:45.492221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:45.492221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:45.492221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:45.492221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:45.492221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:45.492221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:45.492221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:45.492221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:45.492221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:45.492221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:45.492221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:45.493133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:45.493133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:45.493133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:45.493133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:45.496239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 475/977] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:45.502450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909b237-401d-48fd-bed8-64967ef8c669 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:45.548979 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a37f91-4413-40dc-8906-49ecc00af992 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:45.549257 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a37f91-4413-40dc-8906-49ecc00af992 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:45.549530 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a37f91-4413-40dc-8906-49ecc00af992 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:45.550954 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 476/978] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:26:45.556017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6755a2c-af5e-4c03-a129-853ba14985e2 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:45.556306 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6755a2c-af5e-4c03-a129-853ba14985e2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:45.556579 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6755a2c-af5e-4c03-a129-853ba14985e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:45.569319 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:45.575676 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f6755a2c-af5e-4c03-a129-853ba14985e2 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:45.576853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f6755a2c-af5e-4c03-a129-853ba14985e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:45.622204 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:45.627010 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909b237-401d-48fd-bed8-64967ef8c669 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:45.647884 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 477/979] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:26:45.666166 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94e17c-8909-4418-8171-8a2a81916cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:45.666857 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94e17c-8909-4418-8171-8a2a81916cfc None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:45.667309 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94e17c-8909-4418-8171-8a2a81916cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:45.667739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94e17c-8909-4418-8171-8a2a81916cfc None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:45.668465 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94e17c-8909-4418-8171-8a2a81916cfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:45.668783 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94e17c-8909-4418-8171-8a2a81916cfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:45.670343 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94e17c-8909-4418-8171-8a2a81916cfc None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:45.671293 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6755a2c-af5e-4c03-a129-853ba14985e2 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:45.683403 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 503/980] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:45.688782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94e17c-8909-4418-8171-8a2a81916cfc None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:45.784575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6755a2c-af5e-4c03-a129-853ba14985e2 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:45.793020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94e17c-8909-4418-8171-8a2a81916cfc None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:45.806951 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:45.824017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddff831-0ec9-40c8-a75b-288fc6796c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:45.824551 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddff831-0ec9-40c8-a75b-288fc6796c8f None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:45.824834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddff831-0ec9-40c8-a75b-288fc6796c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:45.825116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddff831-0ec9-40c8-a75b-288fc6796c8f None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:45.825864 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddff831-0ec9-40c8-a75b-288fc6796c8f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:45.827230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddff831-0ec9-40c8-a75b-288fc6796c8f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:45.829145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5d927-2355-472f-adbc-ff4bc53409c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:45.833442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5d927-2355-472f-adbc-ff4bc53409c0 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:45.833442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5d927-2355-472f-adbc-ff4bc53409c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:45.833442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5d927-2355-472f-adbc-ff4bc53409c0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:45.837839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5d927-2355-472f-adbc-ff4bc53409c0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:45.839860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5d927-2355-472f-adbc-ff4bc53409c0 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:45.855569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-e4a5d927-2355-472f-adbc-ff4bc53409c0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:26:45.855569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:45.855569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:45.855569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:45.855569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:45.855569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:45.855569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:45.855569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:45.855569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:45.855569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:45.855569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:45.855569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:45.855569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:45.855569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:45.855569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:45.855569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:45.857328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:45.857328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:45.857328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:45.857328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:45.857328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:45.857328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:45.857328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:45.857328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:45.857328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:45.857328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:45.857328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:45.857328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:45.857328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:45.857328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:45.857328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:45.857328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:45.857328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:45.857328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:45.858273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:45.858273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:45.858273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:45.858273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:45.858273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:45.858273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:45.858273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:45.858273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:45.858273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:26:45.858273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:45.861592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 504/981] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jun 23 11:26:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:45.862323 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5d927-2355-472f-adbc-ff4bc53409c0 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:45.900910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 478/982] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 23 11:26:45 2022] POST /identity/v3/services => generated 242 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:26:45.971968 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5d927-2355-472f-adbc-ff4bc53409c0 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:46.016536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0b54d-1a9c-406a-bee1-ddea7d6002d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:46.017232 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0b54d-1a9c-406a-bee1-ddea7d6002d8 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:46.017517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0b54d-1a9c-406a-bee1-ddea7d6002d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:46.017795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0b54d-1a9c-406a-bee1-ddea7d6002d8 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:46.018683 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a0b54d-1a9c-406a-bee1-ddea7d6002d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:46.019003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a0b54d-1a9c-406a-bee1-ddea7d6002d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:46.020476 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a0b54d-1a9c-406a-bee1-ddea7d6002d8 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:46.042721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 505/983] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Jun 23 11:26:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:46.045365 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddff831-0ec9-40c8-a75b-288fc6796c8f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:46.086074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:46.155105 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddff831-0ec9-40c8-a75b-288fc6796c8f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:46.190287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb13ad-163c-4066-af8f-0509275e6676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:46.190973 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb13ad-163c-4066-af8f-0509275e6676 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:46.191257 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb13ad-163c-4066-af8f-0509275e6676 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:46.191531 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb13ad-163c-4066-af8f-0509275e6676 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:46.259459 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fb13ad-163c-4066-af8f-0509275e6676 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:46.261428 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fb13ad-163c-4066-af8f-0509275e6676 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:46.296959 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:46.370897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 479/984] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:26:46 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:26:46.373127 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0b54d-1a9c-406a-bee1-ddea7d6002d8 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:46.402897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:46.474758 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0b54d-1a9c-406a-bee1-ddea7d6002d8 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:46.515879 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944424b-0d94-4e7d-b46a-c9e0c0174014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:46.516676 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944424b-0d94-4e7d-b46a-c9e0c0174014 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:46.516956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944424b-0d94-4e7d-b46a-c9e0c0174014 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:46.517231 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944424b-0d94-4e7d-b46a-c9e0c0174014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:46.527668 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7944424b-0d94-4e7d-b46a-c9e0c0174014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:46.529114 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7944424b-0d94-4e7d-b46a-c9e0c0174014 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:46.547080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-7944424b-0d94-4e7d-b46a-c9e0c0174014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:46.547080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:46.547080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:46.547080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:46.547080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:46.547080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:46.547080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:46.547080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:46.547080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:46.547080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:46.547080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:46.547080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:46.547080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:46.547080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:46.547080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:46.548019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:46.548019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:46.548019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:46.548019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:46.548019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:46.548019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:46.548019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:46.548019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:46.548019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:46.548019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:46.548019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:46.548019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:46.548019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:46.548019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:46.548019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:46.548019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:46.548019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:46.549023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:46.549023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:46.549023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:46.549023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:46.549023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 506/985] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:46.554793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb13ad-163c-4066-af8f-0509275e6676 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:46.655906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb13ad-163c-4066-af8f-0509275e6676 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:46.689222 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c0a25-75a6-4674-8294-021fd50e6e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:46.689888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c0a25-75a6-4674-8294-021fd50e6e82 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:46.690174 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c0a25-75a6-4674-8294-021fd50e6e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:46.690452 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c0a25-75a6-4674-8294-021fd50e6e82 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:46.691195 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c0a25-75a6-4674-8294-021fd50e6e82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:46.691509 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c0a25-75a6-4674-8294-021fd50e6e82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:46.693160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c0a25-75a6-4674-8294-021fd50e6e82 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:46.695557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:46.705872 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 480/986] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:46.711396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944424b-0d94-4e7d-b46a-c9e0c0174014 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:46.789010 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:46.811002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944424b-0d94-4e7d-b46a-c9e0c0174014 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:46.845353 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f84e58-78a6-4338-9394-aca799f1fd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:46.846147 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f84e58-78a6-4338-9394-aca799f1fd48 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:46.846427 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f84e58-78a6-4338-9394-aca799f1fd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:46.846725 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f84e58-78a6-4338-9394-aca799f1fd48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:46.853443 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f84e58-78a6-4338-9394-aca799f1fd48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:46.855555 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f84e58-78a6-4338-9394-aca799f1fd48 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:46.869074 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-f7f84e58-78a6-4338-9394-aca799f1fd48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:46.869074 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:46.869074 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:46.869074 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:46.869074 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:46.869074 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:46.869074 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:46.869074 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:46.869074 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:46.869074 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:46.869074 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:46.869074 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:46.869074 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:46.869074 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:46.869074 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:46.869948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:46.869948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:46.869948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:46.869948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:46.869948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:46.869948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:46.869948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:46.869948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:46.869948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:46.869948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:46.869948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:46.869948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:46.869948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:46.869948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:46.869948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:46.869948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:46.869948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:46.871819 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:46.871819 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:46.871819 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:46.871819 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:46.871819 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:46.871819 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:46.871819 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:46.871819 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:46.874621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 507/987] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:46.876762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c0a25-75a6-4674-8294-021fd50e6e82 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:46.976699 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c0a25-75a6-4674-8294-021fd50e6e82 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:47.013879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933527cd-8963-423b-bdcd-59bea038482a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:47.014563 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933527cd-8963-423b-bdcd-59bea038482a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:47.014844 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933527cd-8963-423b-bdcd-59bea038482a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:47.015115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933527cd-8963-423b-bdcd-59bea038482a None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:47.015890 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933527cd-8963-423b-bdcd-59bea038482a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:47.016193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933527cd-8963-423b-bdcd-59bea038482a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:47.017614 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933527cd-8963-423b-bdcd-59bea038482a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:47.034923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 481/988] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:47.036308 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f84e58-78a6-4338-9394-aca799f1fd48 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:47.142482 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f84e58-78a6-4338-9394-aca799f1fd48 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:47.176578 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49bb34-da4d-41cc-9015-1fc44a2b4651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:47.177229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49bb34-da4d-41cc-9015-1fc44a2b4651 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:47.177529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49bb34-da4d-41cc-9015-1fc44a2b4651 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:47.177815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49bb34-da4d-41cc-9015-1fc44a2b4651 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:47.178663 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49bb34-da4d-41cc-9015-1fc44a2b4651 None admin] RBAC: Adding query filter params (role.id=a9a87bede27348b89fa09b9dfd04e89c, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=030614ef56f143f8be6fcc0445ec9568) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:47.178974 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49bb34-da4d-41cc-9015-1fc44a2b4651 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:47.180387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49bb34-da4d-41cc-9015-1fc44a2b4651 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:47.187484 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 508/989] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:26:47 2022] GET /identity/v3/role_assignments?user.id=030614ef56f143f8be6fcc0445ec9568&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=a9a87bede27348b89fa09b9dfd04e89c => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:47.449737 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb487f-2204-4c94-a89b-b8e0f95e1cf0 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:47.450219 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb487f-2204-4c94-a89b-b8e0f95e1cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:47.450219 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb487f-2204-4c94-a89b-b8e0f95e1cf0 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:47.451758 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 482/990] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:47.456856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71379a1e-5fa7-4746-a9a5-b5d42a5eb406 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:47.457157 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71379a1e-5fa7-4746-a9a5-b5d42a5eb406 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:47.457442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71379a1e-5fa7-4746-a9a5-b5d42a5eb406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:47.462375 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:47.476471 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-71379a1e-5fa7-4746-a9a5-b5d42a5eb406 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:47.486763 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-71379a1e-5fa7-4746-a9a5-b5d42a5eb406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:47.554572 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 509/991] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:47.575781 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb487f-2204-4c94-a89b-b8e0f95e1cf0 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:47.635283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:47.681302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb487f-2204-4c94-a89b-b8e0f95e1cf0 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:47.715851 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85256e-ac68-4d0d-b02b-1e902271d901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:47.716584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85256e-ac68-4d0d-b02b-1e902271d901 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:47.716869 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85256e-ac68-4d0d-b02b-1e902271d901 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:47.717143 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85256e-ac68-4d0d-b02b-1e902271d901 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:47.717800 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d85256e-ac68-4d0d-b02b-1e902271d901 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:47.719201 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d85256e-ac68-4d0d-b02b-1e902271d901 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:47.731984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-7d85256e-ac68-4d0d-b02b-1e902271d901 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:26:47.731984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:47.731984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:47.731984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:47.731984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:47.731984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:47.731984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:47.731984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:47.731984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:26:47.731984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:47.731984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:47.731984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:47.731984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:47.731984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:47.731984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:47.731984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:47.731984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:47.733287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:47.733287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:47.733287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:47.733287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:47.733287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:47.733287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:47.733287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:47.733287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:47.733287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:47.733287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:47.733287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:47.733287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:47.733287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:47.733287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:47.733287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:47.733287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:47.734125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:47.734125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:47.734125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:47.734125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:47.734125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:47.734125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:47.734125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:47.734125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:47.734125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:47.734125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:47.734125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:47.734125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:47.734125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:26:47.734125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:47.737675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 483/992] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 23 11:26:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:47.738925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71379a1e-5fa7-4746-a9a5-b5d42a5eb406 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:47.841929 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71379a1e-5fa7-4746-a9a5-b5d42a5eb406 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:47.872823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:47.878339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ad3bd-e890-4c99-81da-a8519af72ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:47.879047 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ad3bd-e890-4c99-81da-a8519af72ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:47.879363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ad3bd-e890-4c99-81da-a8519af72ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:47.879649 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ad3bd-e890-4c99-81da-a8519af72ae3 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:47.880345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ad3bd-e890-4c99-81da-a8519af72ae3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:47.880662 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ad3bd-e890-4c99-81da-a8519af72ae3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:47.882016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ad3bd-e890-4c99-81da-a8519af72ae3 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:47.890606 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 510/993] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jun 23 11:26:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:47.892852 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85256e-ac68-4d0d-b02b-1e902271d901 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:47.973197 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fd640-c8a0-470f-87b0-28af773e6e53 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:47.973380 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fd640-c8a0-470f-87b0-28af773e6e53 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:47.973667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fd640-c8a0-470f-87b0-28af773e6e53 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:47.975336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 511/994] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:47.980782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e71531-7aa3-4b01-90d8-604b4180133d None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:47.981068 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e71531-7aa3-4b01-90d8-604b4180133d None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:47.981346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e71531-7aa3-4b01-90d8-604b4180133d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:47.998013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-f8e71531-7aa3-4b01-90d8-604b4180133d None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:47.999230 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f8e71531-7aa3-4b01-90d8-604b4180133d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:47.999448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85256e-ac68-4d0d-b02b-1e902271d901 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:48.019220 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:48.033908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0405a34-07db-4f26-a52b-ae74f994b449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:48.034588 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0405a34-07db-4f26-a52b-ae74f994b449 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:48.034884 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0405a34-07db-4f26-a52b-ae74f994b449 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:48.035152 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0405a34-07db-4f26-a52b-ae74f994b449 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:48.035861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0405a34-07db-4f26-a52b-ae74f994b449 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:48.036169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0405a34-07db-4f26-a52b-ae74f994b449 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:48.037531 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0405a34-07db-4f26-a52b-ae74f994b449 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:48.043114 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 484/995] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jun 23 11:26:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:48.053271 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0405a34-07db-4f26-a52b-ae74f994b449 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:48.071680 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 512/996] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:26:48.094651 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e71531-7aa3-4b01-90d8-604b4180133d None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:48.155586 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0405a34-07db-4f26-a52b-ae74f994b449 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:48.192889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541318a3-e4c0-45d0-b766-d235ccfb4540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:48.193665 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541318a3-e4c0-45d0-b766-d235ccfb4540 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:48.193949 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541318a3-e4c0-45d0-b766-d235ccfb4540 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:48.194225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541318a3-e4c0-45d0-b766-d235ccfb4540 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:48.194989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541318a3-e4c0-45d0-b766-d235ccfb4540 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:48.196308 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e71531-7aa3-4b01-90d8-604b4180133d None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:48.196412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541318a3-e4c0-45d0-b766-d235ccfb4540 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:48.222258 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:48.230851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ce4c2-ca4a-4d61-b4f8-5a28ee713bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:48.231584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ce4c2-ca4a-4d61-b4f8-5a28ee713bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:48.231866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ce4c2-ca4a-4d61-b4f8-5a28ee713bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:48.232136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ce4c2-ca4a-4d61-b4f8-5a28ee713bf7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:48.232788 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141ce4c2-ca4a-4d61-b4f8-5a28ee713bf7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:48.234148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141ce4c2-ca4a-4d61-b4f8-5a28ee713bf7 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:48.246210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-141ce4c2-ca4a-4d61-b4f8-5a28ee713bf7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 11:26:48.246210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:48.246210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:48.246210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:48.246210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:48.246210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:48.246210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:48.246210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:48.246210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 11:26:48.246210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:48.246210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:48.246210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:48.246210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:48.246210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:48.246210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:48.246210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:48.246210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:48.247305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:48.247305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:48.247305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:48.247305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:48.247305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:48.247305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:48.247305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:48.247305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:48.247305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:48.247305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:48.247305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:48.247305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:48.247305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:48.247305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:48.247305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:48.247305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:48.248163 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:48.248163 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:48.248163 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:48.248163 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:48.248163 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:48.248163 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:48.248163 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:48.248163 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:48.248163 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:48.248163 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:48.248163 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:48.248163 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:48.248163 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 11:26:48.248163 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:48.248163 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 513/997] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:48.253137 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ce4c2-ca4a-4d61-b4f8-5a28ee713bf7 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:48.305780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 485/998] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jun 23 11:26:48 2022] POST /identity/v3/endpoints => generated 325 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:26:48.362938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ce4c2-ca4a-4d61-b4f8-5a28ee713bf7 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:48.408803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d11ec5-bb8a-4e54-ae92-8c4bf5a97b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:48.409486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d11ec5-bb8a-4e54-ae92-8c4bf5a97b61 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:48.409820 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d11ec5-bb8a-4e54-ae92-8c4bf5a97b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:48.410100 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d11ec5-bb8a-4e54-ae92-8c4bf5a97b61 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:48.410850 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d11ec5-bb8a-4e54-ae92-8c4bf5a97b61 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:48.411160 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d11ec5-bb8a-4e54-ae92-8c4bf5a97b61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:48.412607 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d11ec5-bb8a-4e54-ae92-8c4bf5a97b61 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:48.423412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 514/999] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 23 11:26:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:48.424308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541318a3-e4c0-45d0-b766-d235ccfb4540 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:48.475603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:48.538983 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541318a3-e4c0-45d0-b766-d235ccfb4540 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:48.574834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975655e-c566-43b8-b279-4e009ed07f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:48.575506 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975655e-c566-43b8-b279-4e009ed07f58 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:48.575790 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975655e-c566-43b8-b279-4e009ed07f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:48.576066 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975655e-c566-43b8-b279-4e009ed07f58 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:48.576770 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6975655e-c566-43b8-b279-4e009ed07f58 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:48.577078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6975655e-c566-43b8-b279-4e009ed07f58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:48.578444 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6975655e-c566-43b8-b279-4e009ed07f58 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:48.583594 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 486/1000] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 23 11:26:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:48.589435 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d11ec5-bb8a-4e54-ae92-8c4bf5a97b61 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:48.641829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:48.691706 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d11ec5-bb8a-4e54-ae92-8c4bf5a97b61 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:48.724845 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfd801-f086-4f70-aa30-5d7c3c194299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:48.725502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfd801-f086-4f70-aa30-5d7c3c194299 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:48.725786 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfd801-f086-4f70-aa30-5d7c3c194299 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:48.726058 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfd801-f086-4f70-aa30-5d7c3c194299 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:48.726871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bfd801-f086-4f70-aa30-5d7c3c194299 None admin] RBAC: Adding query filter params (interface=public, service_id=66bc399878d743d897733beae0a65747, region_id=RegionOne) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:48.727183 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bfd801-f086-4f70-aa30-5d7c3c194299 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:48.728545 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bfd801-f086-4f70-aa30-5d7c3c194299 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:48.738887 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 515/1001] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jun 23 11:26:48 2022] GET /identity/v3/endpoints?service_id=66bc399878d743d897733beae0a65747&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:48.740353 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975655e-c566-43b8-b279-4e009ed07f58 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:48.844388 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975655e-c566-43b8-b279-4e009ed07f58 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:48.853548 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:48.878890 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb7361-6acc-469f-8f3f-083e0be43bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:48.879560 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb7361-6acc-469f-8f3f-083e0be43bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:48.879843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb7361-6acc-469f-8f3f-083e0be43bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:48.880118 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb7361-6acc-469f-8f3f-083e0be43bc8 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:48.880804 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb7361-6acc-469f-8f3f-083e0be43bc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:48.882168 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb7361-6acc-469f-8f3f-083e0be43bc8 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:48.888605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 487/1002] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jun 23 11:26:48 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:26:49.324995 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7119ca6-24d1-498c-9460-b492e5806f79 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:49.325499 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7119ca6-24d1-498c-9460-b492e5806f79 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:49.325576 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7119ca6-24d1-498c-9460-b492e5806f79 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:49.327138 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 516/1003] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:49.332105 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974a5ee-4b8e-4497-a0fd-fd7c4ee7b167 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:49.332404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974a5ee-4b8e-4497-a0fd-fd7c4ee7b167 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:49.332687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974a5ee-4b8e-4497-a0fd-fd7c4ee7b167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:49.349813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d974a5ee-4b8e-4497-a0fd-fd7c4ee7b167 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:49.351018 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d974a5ee-4b8e-4497-a0fd-fd7c4ee7b167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:49.425378 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 488/1004] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:49 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:49.448873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7119ca6-24d1-498c-9460-b492e5806f79 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:49.562481 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7119ca6-24d1-498c-9460-b492e5806f79 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:49.577470 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:49.597765 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161a9f8-fc94-4718-b42f-e1e6de3dc4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:49.598576 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161a9f8-fc94-4718-b42f-e1e6de3dc4bd None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:49.598867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161a9f8-fc94-4718-b42f-e1e6de3dc4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:49.599149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161a9f8-fc94-4718-b42f-e1e6de3dc4bd None admin] PATH_INFO: `/v3/services/image` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:49.599798 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3161a9f8-fc94-4718-b42f-e1e6de3dc4bd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:49.601164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3161a9f8-fc94-4718-b42f-e1e6de3dc4bd None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:49.613325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-3161a9f8-fc94-4718-b42f-e1e6de3dc4bd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 11:26:49.613325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:49.613325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:49.613325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:49.613325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:49.613325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:49.613325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:49.613325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:49.613325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 11:26:49.613325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:49.613325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:49.613325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:49.613325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:49.613325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:49.613325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:49.613325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:49.613325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:49.614502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:49.614502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:49.614502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:49.614502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:49.614502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:49.614502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:49.614502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:49.614502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:49.614502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:49.614502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:49.614502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:49.614502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:49.614502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:49.614502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:49.614502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:49.614502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:49.615729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:49.615729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:49.615729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:49.615729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:49.615729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:49.615729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:49.615729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:49.615729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:49.615729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:49.615729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:49.615729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:49.615729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:49.615729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 11:26:49.615729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:49.618444 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 517/1005] 10.222.0.13 () {56 vars in 1105 bytes} [Thu Jun 23 11:26:49 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:49.624974 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974a5ee-4b8e-4497-a0fd-fd7c4ee7b167 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:49.723808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974a5ee-4b8e-4497-a0fd-fd7c4ee7b167 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:49.725366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:49.759731 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a2100-e2ab-4704-8581-1d8091e32c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:49.761425 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a2100-e2ab-4704-8581-1d8091e32c0b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:49.761744 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a2100-e2ab-4704-8581-1d8091e32c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:49.762021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a2100-e2ab-4704-8581-1d8091e32c0b None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:49.762755 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a2100-e2ab-4704-8581-1d8091e32c0b None admin] RBAC: Adding query filter params (name=image) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:49.763064 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a2100-e2ab-4704-8581-1d8091e32c0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:49.764498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a2100-e2ab-4704-8581-1d8091e32c0b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:49.769583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 489/1006] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:49.775001 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161a9f8-fc94-4718-b42f-e1e6de3dc4bd None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:49.876729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161a9f8-fc94-4718-b42f-e1e6de3dc4bd None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:49.910198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de56a3-7a1e-4cb3-bf45-1e733ffcb509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:49.910888 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de56a3-7a1e-4cb3-bf45-1e733ffcb509 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:49.911181 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de56a3-7a1e-4cb3-bf45-1e733ffcb509 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:49.911449 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de56a3-7a1e-4cb3-bf45-1e733ffcb509 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:49.912153 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de56a3-7a1e-4cb3-bf45-1e733ffcb509 None admin] RBAC: Adding query filter params (type=image) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:49.912520 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de56a3-7a1e-4cb3-bf45-1e733ffcb509 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:49.913939 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de56a3-7a1e-4cb3-bf45-1e733ffcb509 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:49.919368 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 518/1007] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:49 2022] GET /identity/v3/services?type=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:50.398446 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a021b-baf1-4bf8-b34f-ea488f60ecce None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:50.398831 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a021b-baf1-4bf8-b34f-ea488f60ecce None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:50.399049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a021b-baf1-4bf8-b34f-ea488f60ecce None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:50.400570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 490/1008] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:50.405702 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d85206-b1b6-4759-b5d1-0ab535080623 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:50.406152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d85206-b1b6-4759-b5d1-0ab535080623 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:50.406315 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d85206-b1b6-4759-b5d1-0ab535080623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:50.422865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-f2d85206-b1b6-4759-b5d1-0ab535080623 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:50.424062 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f2d85206-b1b6-4759-b5d1-0ab535080623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:50.464931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:50.506668 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 519/1009] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:50 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:50.529210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a021b-baf1-4bf8-b34f-ea488f60ecce None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:50.634435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a021b-baf1-4bf8-b34f-ea488f60ecce None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:50.669537 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46290297-0518-48ff-931e-82ce87ee9fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:50.670255 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46290297-0518-48ff-931e-82ce87ee9fca None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:50.670562 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46290297-0518-48ff-931e-82ce87ee9fca None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:50.670846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46290297-0518-48ff-931e-82ce87ee9fca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:50.672030 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46290297-0518-48ff-931e-82ce87ee9fca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:50.673465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46290297-0518-48ff-931e-82ce87ee9fca None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:50.679842 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 491/1010] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:50.681079 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d85206-b1b6-4759-b5d1-0ab535080623 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:50.782243 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d85206-b1b6-4759-b5d1-0ab535080623 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:50.817303 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d33d1-87f4-4af0-b950-897134785007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:50.817962 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d33d1-87f4-4af0-b950-897134785007 None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:50.818247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d33d1-87f4-4af0-b950-897134785007 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:50.818525 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d33d1-87f4-4af0-b950-897134785007 None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:50.819285 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d33d1-87f4-4af0-b950-897134785007 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:50.820725 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d33d1-87f4-4af0-b950-897134785007 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:50.906558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 520/1011] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 23 11:26:50 2022] POST /identity/v3/projects => generated 311 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:26:50.930469 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440154a-ea9a-4b90-b5b2-8c0a77208ca9 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:50.930828 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440154a-ea9a-4b90-b5b2-8c0a77208ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:50.930963 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440154a-ea9a-4b90-b5b2-8c0a77208ca9 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:50.932540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 492/1012] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:50.938667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c952a7-7c78-4dee-8738-47cddba82d43 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:50.946089 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c952a7-7c78-4dee-8738-47cddba82d43 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:50.946089 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c952a7-7c78-4dee-8738-47cddba82d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:50.957745 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-82c952a7-7c78-4dee-8738-47cddba82d43 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:50.959026 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-82c952a7-7c78-4dee-8738-47cddba82d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:50.979039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:51.038585 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 521/1013] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:50 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:51.060245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440154a-ea9a-4b90-b5b2-8c0a77208ca9 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:51.100274 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:51.171971 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440154a-ea9a-4b90-b5b2-8c0a77208ca9 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:51.207682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ebaf5-15e6-4bb6-a960-e8f3e01b4df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:51.208402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ebaf5-15e6-4bb6-a960-e8f3e01b4df5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:51.208687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ebaf5-15e6-4bb6-a960-e8f3e01b4df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:51.208964 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ebaf5-15e6-4bb6-a960-e8f3e01b4df5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:51.209677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ebaf5-15e6-4bb6-a960-e8f3e01b4df5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:51.211142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ebaf5-15e6-4bb6-a960-e8f3e01b4df5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:51.223893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-f90ebaf5-15e6-4bb6-a960-e8f3e01b4df5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 11:26:51.223893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:51.223893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:51.223893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:51.223893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:51.223893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:51.223893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:51.223893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:51.223893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 11:26:51.223893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:51.223893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:51.223893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:51.223893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:51.223893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:51.223893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:51.223893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:51.223893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:51.224893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:51.224893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:51.224893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:51.224893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:51.224893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:51.224893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:51.224893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:51.224893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:51.224893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:51.224893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:51.224893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:51.224893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:51.224893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:51.224893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:51.224893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:51.224893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:51.225734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:51.225734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:51.225734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:51.225734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:51.225734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:51.225734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:51.225734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:51.225734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:51.225734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:51.225734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:51.225734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:51.225734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:51.225734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 11:26:51.225734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:51.225734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 493/1014] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:51.231797 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c952a7-7c78-4dee-8738-47cddba82d43 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:51.333589 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c952a7-7c78-4dee-8738-47cddba82d43 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:51.368845 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ca742-da0e-4b1e-9417-75fb2bdec2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:51.369516 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ca742-da0e-4b1e-9417-75fb2bdec2fb None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:51.369801 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ca742-da0e-4b1e-9417-75fb2bdec2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:51.370225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ca742-da0e-4b1e-9417-75fb2bdec2fb None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:51.371138 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ca742-da0e-4b1e-9417-75fb2bdec2fb None admin] RBAC: Adding query filter params (name=compute) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:51.371454 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ca742-da0e-4b1e-9417-75fb2bdec2fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:51.372829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ca742-da0e-4b1e-9417-75fb2bdec2fb None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:51.377931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 522/1015] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 23 11:26:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:51.383344 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ebaf5-15e6-4bb6-a960-e8f3e01b4df5 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:51.491590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:51.496500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ebaf5-15e6-4bb6-a960-e8f3e01b4df5 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:51.531207 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d40166-925b-426a-8552-1992555dfc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:51.532375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d40166-925b-426a-8552-1992555dfc36 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:51.532914 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d40166-925b-426a-8552-1992555dfc36 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:51.533501 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d40166-925b-426a-8552-1992555dfc36 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:51.534456 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d40166-925b-426a-8552-1992555dfc36 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:51.535285 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d40166-925b-426a-8552-1992555dfc36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:51.536661 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d40166-925b-426a-8552-1992555dfc36 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:51.541816 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 494/1016] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 23 11:26:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:51.547353 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ca742-da0e-4b1e-9417-75fb2bdec2fb None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:51.567360 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:51.649867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ca742-da0e-4b1e-9417-75fb2bdec2fb None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:51.684845 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ba73f-b3aa-494a-a619-3c075c77cdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:51.685506 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ba73f-b3aa-494a-a619-3c075c77cdc7 None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:51.685785 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ba73f-b3aa-494a-a619-3c075c77cdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:51.686055 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ba73f-b3aa-494a-a619-3c075c77cdc7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:51.686833 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ba73f-b3aa-494a-a619-3c075c77cdc7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:51.688201 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ba73f-b3aa-494a-a619-3c075c77cdc7 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:51.774647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 523/1017] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jun 23 11:26:51 2022] POST /identity/v3/endpoints => generated 334 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:26:52.028095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4285d-9828-4d09-91fc-c0f0708da482 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:52.028595 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4285d-9828-4d09-91fc-c0f0708da482 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:52.028595 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4285d-9828-4d09-91fc-c0f0708da482 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:52.029934 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 495/1018] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:52.035492 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e60771-588f-48e0-ac38-8ca57ae3fed7 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:52.035787 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e60771-588f-48e0-ac38-8ca57ae3fed7 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:52.036072 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e60771-588f-48e0-ac38-8ca57ae3fed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:52.053153 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-16e60771-588f-48e0-ac38-8ca57ae3fed7 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:52.054321 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-16e60771-588f-48e0-ac38-8ca57ae3fed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:52.125343 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:52.130978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 524/1019] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:52 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:52.155460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4285d-9828-4d09-91fc-c0f0708da482 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:52.266918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4285d-9828-4d09-91fc-c0f0708da482 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:52.301633 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc035726-7e1e-47fe-8171-9e7111b482d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:52.302217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc035726-7e1e-47fe-8171-9e7111b482d4 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:52.302520 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc035726-7e1e-47fe-8171-9e7111b482d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:52.302816 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc035726-7e1e-47fe-8171-9e7111b482d4 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:52.303548 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc035726-7e1e-47fe-8171-9e7111b482d4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:52.304897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc035726-7e1e-47fe-8171-9e7111b482d4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:52.387791 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 496/1020] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 23 11:26:52 2022] POST /identity/v3/services => generated 242 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:26:53.011809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54af22-bee4-4809-8e05-be3a077e4801 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:53.012364 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54af22-bee4-4809-8e05-be3a077e4801 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:53.012364 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54af22-bee4-4809-8e05-be3a077e4801 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:53.013878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 525/1021] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:53.018900 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6908db0-4568-4d41-ab3e-de211cb0b68a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:53.019194 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6908db0-4568-4d41-ab3e-de211cb0b68a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:53.019473 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6908db0-4568-4d41-ab3e-de211cb0b68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:53.036784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-e6908db0-4568-4d41-ab3e-de211cb0b68a None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:53.037966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e6908db0-4568-4d41-ab3e-de211cb0b68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:53.055487 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:53.113975 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 497/1022] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:53.139053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54af22-bee4-4809-8e05-be3a077e4801 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:53.240975 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54af22-bee4-4809-8e05-be3a077e4801 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:53.273846 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d1c4a-fdae-4ab5-91ee-9ada19484d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:53.274579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d1c4a-fdae-4ab5-91ee-9ada19484d7d None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:53.274864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d1c4a-fdae-4ab5-91ee-9ada19484d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:53.275135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d1c4a-fdae-4ab5-91ee-9ada19484d7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:53.276331 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d1c4a-fdae-4ab5-91ee-9ada19484d7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:53.277745 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d1c4a-fdae-4ab5-91ee-9ada19484d7d None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:53.279512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 526/1023] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:53.284950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6908db0-4568-4d41-ab3e-de211cb0b68a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:53.383498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6908db0-4568-4d41-ab3e-de211cb0b68a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:53.417327 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd4cc0-0dfc-45b9-b249-8bc751674fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:53.417996 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd4cc0-0dfc-45b9-b249-8bc751674fc9 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:53.418276 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd4cc0-0dfc-45b9-b249-8bc751674fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:53.418561 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd4cc0-0dfc-45b9-b249-8bc751674fc9 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:53.419281 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd4cc0-0dfc-45b9-b249-8bc751674fc9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:53.420696 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd4cc0-0dfc-45b9-b249-8bc751674fc9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:53.518102 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 498/1024] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 23 11:26:53 2022] POST /identity/v3/users => generated 285 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:26:53.946468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9fd5a-3c70-435a-97a4-d8bfc3bd86e9 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:53.946882 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9fd5a-3c70-435a-97a4-d8bfc3bd86e9 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:53.947091 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9fd5a-3c70-435a-97a4-d8bfc3bd86e9 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:53.949062 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 527/1025] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:53.954175 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48bad2-a415-4301-808c-daafbfac6e18 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:53.954474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48bad2-a415-4301-808c-daafbfac6e18 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:53.954783 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48bad2-a415-4301-808c-daafbfac6e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:53.961720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:53.972748 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5c48bad2-a415-4301-808c-daafbfac6e18 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:53.973990 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5c48bad2-a415-4301-808c-daafbfac6e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:54.044691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 499/1026] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:54.069156 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9fd5a-3c70-435a-97a4-d8bfc3bd86e9 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:54.139924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:54.176071 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9fd5a-3c70-435a-97a4-d8bfc3bd86e9 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:54.209633 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab44593-0222-4b9e-b27a-eaf63c409640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:54.210362 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab44593-0222-4b9e-b27a-eaf63c409640 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:54.210676 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab44593-0222-4b9e-b27a-eaf63c409640 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:54.210954 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab44593-0222-4b9e-b27a-eaf63c409640 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:54.221851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab44593-0222-4b9e-b27a-eaf63c409640 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:54.231035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-bab44593-0222-4b9e-b27a-eaf63c409640 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:26:54.231035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:54.231035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:54.231035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:54.231035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:54.231035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:54.231035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:54.231035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:54.231035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:54.231035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:54.231035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:54.231035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:54.231035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:54.231035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:54.231035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:54.231035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:54.231956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:54.231956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:54.231956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:54.231956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:54.231956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:54.231956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:54.231956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:54.231956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:54.231956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:54.231956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:54.231956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:54.231956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:54.231956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:54.231956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:54.231956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:54.231956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:54.232831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:54.232831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:54.232831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:54.232831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:54.232831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:54.232831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:54.232831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:54.232831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:54.232831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:54.232831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:54.232831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:26:54.232831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:54.234087 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 528/1027] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Jun 23 11:26:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:54.238134 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48bad2-a415-4301-808c-daafbfac6e18 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:54.341359 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48bad2-a415-4301-808c-daafbfac6e18 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:54.375146 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab28a67-2e1b-4940-ba21-49113c344266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:54.375815 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab28a67-2e1b-4940-ba21-49113c344266 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:54.376095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab28a67-2e1b-4940-ba21-49113c344266 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:54.376365 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab28a67-2e1b-4940-ba21-49113c344266 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:54.377111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab28a67-2e1b-4940-ba21-49113c344266 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:54.377415 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab28a67-2e1b-4940-ba21-49113c344266 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:54.378798 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab28a67-2e1b-4940-ba21-49113c344266 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:54.380135 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:54.392945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 500/1028] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jun 23 11:26:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:54.394331 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab44593-0222-4b9e-b27a-eaf63c409640 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:54.446621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ab799-1bd9-4fcd-8391-5143eb33f771 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:54.446932 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ab799-1bd9-4fcd-8391-5143eb33f771 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:54.447191 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ab799-1bd9-4fcd-8391-5143eb33f771 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:54.448825 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 501/1029] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:26:54.453620 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae85dd2-eb1a-4f60-8a4b-74d129a28132 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:54.453903 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae85dd2-eb1a-4f60-8a4b-74d129a28132 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:54.454190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae85dd2-eb1a-4f60-8a4b-74d129a28132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:54.470504 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-eae85dd2-eb1a-4f60-8a4b-74d129a28132 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:54.471644 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-eae85dd2-eb1a-4f60-8a4b-74d129a28132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:54.505931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab44593-0222-4b9e-b27a-eaf63c409640 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:54.533604 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:54.542883 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b6557-450f-4907-aeb9-c00f5b81e6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:54.545296 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b6557-450f-4907-aeb9-c00f5b81e6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:54.545296 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b6557-450f-4907-aeb9-c00f5b81e6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:54.545296 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b6557-450f-4907-aeb9-c00f5b81e6f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:54.546906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 502/1030] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:26:54.555812 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b6557-450f-4907-aeb9-c00f5b81e6f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:54.557291 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b6557-450f-4907-aeb9-c00f5b81e6f2 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:54.569104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae85dd2-eb1a-4f60-8a4b-74d129a28132 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:54.576383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-b03b6557-450f-4907-aeb9-c00f5b81e6f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:54.576383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:54.576383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:54.576383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:54.576383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:54.576383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:54.576383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:54.576383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:54.576383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:54.576383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:54.576383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:54.576383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:54.576383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:54.576383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:54.576383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:54.577289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:54.577289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:54.577289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:54.577289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:54.577289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:54.577289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:54.577289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:54.577289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:54.577289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:54.577289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:54.577289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:54.577289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:54.577289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:54.577289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:54.577289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:54.577289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:54.577289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:54.578233 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:54.578233 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:54.578233 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:54.578233 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:54.582096 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 529/1031] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:54.582914 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b6557-450f-4907-aeb9-c00f5b81e6f2 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:54.684266 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae85dd2-eb1a-4f60-8a4b-74d129a28132 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:54.687641 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b6557-450f-4907-aeb9-c00f5b81e6f2 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:54.712800 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:54.722986 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd9123-2fb6-4aaf-9913-0a129720419b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:54.723145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33486a-d8c9-4461-884e-6b9d4583f05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:54.723256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd9123-2fb6-4aaf-9913-0a129720419b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:54.723256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd9123-2fb6-4aaf-9913-0a129720419b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:54.723407 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33486a-d8c9-4461-884e-6b9d4583f05c None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:54.723477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd9123-2fb6-4aaf-9913-0a129720419b None admin] PATH_INFO: `/v3/services/image` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:54.723547 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33486a-d8c9-4461-884e-6b9d4583f05c None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:54.723805 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33486a-d8c9-4461-884e-6b9d4583f05c None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:54.724048 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd9123-2fb6-4aaf-9913-0a129720419b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:54.724551 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33486a-d8c9-4461-884e-6b9d4583f05c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:54.724858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33486a-d8c9-4461-884e-6b9d4583f05c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:54.725418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd9123-2fb6-4aaf-9913-0a129720419b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:54.726197 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33486a-d8c9-4461-884e-6b9d4583f05c None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:54.737465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-f9dd9123-2fb6-4aaf-9913-0a129720419b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 11:26:54.737465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:54.737465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:54.737465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:54.737465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:54.737465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:54.737465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:54.737465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:54.737465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 11:26:54.737465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:54.737465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:54.737465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:54.737465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:54.737465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:54.737465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:54.737465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:54.737465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:54.738474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:54.738474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:54.738474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:54.738474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:54.738474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:54.738474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:54.738474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:54.738474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:54.738474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:54.738474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:54.738474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:54.738474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:54.738474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:54.738474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:54.738474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:54.738474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:54.739527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:54.739527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:54.739527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:54.739527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:54.739527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:54.739527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:54.739527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:54.739527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:54.739527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:54.739527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:54.739527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:54.739527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:54.739527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 11:26:54.739527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:54.739527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 503/1032] 10.222.0.13 () {56 vars in 1105 bytes} [Thu Jun 23 11:26:54 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:54.747752 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 530/1033] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:54.747934 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd9123-2fb6-4aaf-9913-0a129720419b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:54.749359 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33486a-d8c9-4461-884e-6b9d4583f05c None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:54.796089 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:54.853073 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd9123-2fb6-4aaf-9913-0a129720419b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:54.857251 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33486a-d8c9-4461-884e-6b9d4583f05c None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:54.887741 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d268c-8d3a-48e2-ae8e-6db5f0f3522d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:54.888390 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d268c-8d3a-48e2-ae8e-6db5f0f3522d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:54.888692 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d268c-8d3a-48e2-ae8e-6db5f0f3522d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:54.888936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d268c-8d3a-48e2-ae8e-6db5f0f3522d None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:54.889643 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d268c-8d3a-48e2-ae8e-6db5f0f3522d None admin] RBAC: Adding query filter params (name=image) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:54.889933 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d268c-8d3a-48e2-ae8e-6db5f0f3522d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:54.891307 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d268c-8d3a-48e2-ae8e-6db5f0f3522d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:54.894810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fedcb-c4e3-427e-a350-7fa7880c73e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:54.895594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fedcb-c4e3-427e-a350-7fa7880c73e5 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:54.895879 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fedcb-c4e3-427e-a350-7fa7880c73e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:54.896154 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fedcb-c4e3-427e-a350-7fa7880c73e5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:54.901523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 504/1034] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:54.902635 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d268c-8d3a-48e2-ae8e-6db5f0f3522d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:54.908551 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484fedcb-c4e3-427e-a350-7fa7880c73e5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:54.909997 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484fedcb-c4e3-427e-a350-7fa7880c73e5 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:54.924294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-484fedcb-c4e3-427e-a350-7fa7880c73e5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:26:54.924294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:54.924294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:54.924294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:54.924294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:54.924294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:54.924294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:54.924294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:54.924294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:54.924294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:54.924294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:54.924294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:54.924294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:54.924294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:54.924294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:54.924294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:54.925414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:54.925414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:54.925414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:54.925414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:54.925414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:54.925414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:54.925414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:54.925414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:54.925414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:54.925414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:54.925414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:54.925414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:54.925414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:54.925414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:54.925414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:54.925414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:54.925414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:54.925414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:54.926406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:54.926406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:54.926406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:54.926406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:54.926406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:54.926406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:54.926406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:54.926406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:54.926406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:26:54.926406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:54.926406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 531/1035] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jun 23 11:26:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:54.931462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fedcb-c4e3-427e-a350-7fa7880c73e5 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:54.966316 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:55.013552 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d268c-8d3a-48e2-ae8e-6db5f0f3522d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:55.033045 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:55.036274 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fedcb-c4e3-427e-a350-7fa7880c73e5 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:55.049046 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfbc38-c643-43ae-bce8-0e336897499f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:55.050317 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfbc38-c643-43ae-bce8-0e336897499f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:55.050831 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfbc38-c643-43ae-bce8-0e336897499f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:55.051118 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfbc38-c643-43ae-bce8-0e336897499f None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:55.051832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bfbc38-c643-43ae-bce8-0e336897499f None admin] RBAC: Adding query filter params (type=image) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:55.052147 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bfbc38-c643-43ae-bce8-0e336897499f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:55.053524 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bfbc38-c643-43ae-bce8-0e336897499f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:55.063298 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 505/1036] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:55.065298 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfbc38-c643-43ae-bce8-0e336897499f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:55.071997 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152034eb-3dc1-434d-a4a2-aee472a706a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:55.072665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152034eb-3dc1-434d-a4a2-aee472a706a1 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:55.073001 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152034eb-3dc1-434d-a4a2-aee472a706a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:55.073274 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152034eb-3dc1-434d-a4a2-aee472a706a1 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:55.074136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152034eb-3dc1-434d-a4a2-aee472a706a1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:55.074447 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152034eb-3dc1-434d-a4a2-aee472a706a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:55.075901 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152034eb-3dc1-434d-a4a2-aee472a706a1 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:55.097635 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 532/1037] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jun 23 11:26:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:55.098647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152034eb-3dc1-434d-a4a2-aee472a706a1 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:55.178954 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfbc38-c643-43ae-bce8-0e336897499f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:55.196799 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:55.205148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152034eb-3dc1-434d-a4a2-aee472a706a1 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:55.217059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb76aa0-fec7-462f-96d4-b70cb2dd9189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:55.217725 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb76aa0-fec7-462f-96d4-b70cb2dd9189 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:55.218010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb76aa0-fec7-462f-96d4-b70cb2dd9189 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:55.218287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb76aa0-fec7-462f-96d4-b70cb2dd9189 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:55.219121 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb76aa0-fec7-462f-96d4-b70cb2dd9189 None admin] RBAC: Adding query filter params (interface=public, service_id=7df12bf53e3649099bd8bf67eee8eab5, region_id=RegionOne) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:55.219434 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb76aa0-fec7-462f-96d4-b70cb2dd9189 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:55.220797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb76aa0-fec7-462f-96d4-b70cb2dd9189 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:55.226384 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 506/1038] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jun 23 11:26:55 2022] GET /identity/v3/endpoints?service_id=7df12bf53e3649099bd8bf67eee8eab5&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:55.231831 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb76aa0-fec7-462f-96d4-b70cb2dd9189 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:55.241652 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de9779-4658-4f6a-9eff-606233c1d0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:55.242299 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de9779-4658-4f6a-9eff-606233c1d0bf None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:55.242613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de9779-4658-4f6a-9eff-606233c1d0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:55.242889 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de9779-4658-4f6a-9eff-606233c1d0bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:55.271610 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:55.315401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7de9779-4658-4f6a-9eff-606233c1d0bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:55.317165 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7de9779-4658-4f6a-9eff-606233c1d0bf None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:55.335224 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb76aa0-fec7-462f-96d4-b70cb2dd9189 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:55.374457 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b998f-9d8e-4872-ab4f-725e3c78323f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:55.375400 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b998f-9d8e-4872-ab4f-725e3c78323f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:55.375686 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b998f-9d8e-4872-ab4f-725e3c78323f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:55.376113 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b998f-9d8e-4872-ab4f-725e3c78323f None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:55.377100 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b998f-9d8e-4872-ab4f-725e3c78323f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:55.378462 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b998f-9d8e-4872-ab4f-725e3c78323f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:55.386873 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 507/1039] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jun 23 11:26:55 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:26:55.424779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 533/1040] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:26:55 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 11:26:55.430898 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b998f-9d8e-4872-ab4f-725e3c78323f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:55.495105 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:55.564370 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b998f-9d8e-4872-ab4f-725e3c78323f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:55.599459 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f2af2-4f3a-4265-a1d1-8ab27a2c1dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:55.600345 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f2af2-4f3a-4265-a1d1-8ab27a2c1dfa None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:55.600632 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f2af2-4f3a-4265-a1d1-8ab27a2c1dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:55.600904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f2af2-4f3a-4265-a1d1-8ab27a2c1dfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:55.611996 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f2af2-4f3a-4265-a1d1-8ab27a2c1dfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:55.613435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f2af2-4f3a-4265-a1d1-8ab27a2c1dfa None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:55.631491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-d96f2af2-4f3a-4265-a1d1-8ab27a2c1dfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:55.631491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:55.631491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:55.631491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:55.631491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:55.631491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:55.631491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:55.631491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:55.631491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:55.631491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:55.631491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:55.631491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:55.631491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:55.631491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:55.631491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:55.632439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:55.632439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:55.632439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:55.632439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:55.632439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:55.632439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:55.632439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:55.632439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:55.632439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:55.632439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:55.632439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:55.632439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:55.632439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:55.632439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:55.632439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:55.632439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:55.632439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:55.638824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:55.638824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:55.638824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:55.638824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:55.638824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 508/1041] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:55.640932 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de9779-4658-4f6a-9eff-606233c1d0bf None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:55.641512 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfa7af-d3f0-49bf-8054-3000bc3c1913 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:55.641799 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfa7af-d3f0-49bf-8054-3000bc3c1913 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:55.642073 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfa7af-d3f0-49bf-8054-3000bc3c1913 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:55.643675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 509/1042] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:26:55.648551 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b2d73-d4bb-4385-826b-001966f6b8af None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:55.648830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b2d73-d4bb-4385-826b-001966f6b8af None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:55.649099 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b2d73-d4bb-4385-826b-001966f6b8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:55.666335 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f24b2d73-d4bb-4385-826b-001966f6b8af None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:55.667797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f24b2d73-d4bb-4385-826b-001966f6b8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:55.703664 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:55.741228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 510/1043] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:26:55.746744 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de9779-4658-4f6a-9eff-606233c1d0bf None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:55.765969 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b2d73-d4bb-4385-826b-001966f6b8af None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:55.781312 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8530bc2-98f1-4a5b-9bf2-40dd2a2212cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:55.781976 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8530bc2-98f1-4a5b-9bf2-40dd2a2212cd None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:55.782260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8530bc2-98f1-4a5b-9bf2-40dd2a2212cd None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:55.782563 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8530bc2-98f1-4a5b-9bf2-40dd2a2212cd None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:55.783277 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8530bc2-98f1-4a5b-9bf2-40dd2a2212cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:55.783584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8530bc2-98f1-4a5b-9bf2-40dd2a2212cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:55.784982 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8530bc2-98f1-4a5b-9bf2-40dd2a2212cd None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:55.796097 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:55.804030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 534/1044] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:55.805342 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8530bc2-98f1-4a5b-9bf2-40dd2a2212cd None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:55.876929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b2d73-d4bb-4385-826b-001966f6b8af None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:55.911293 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3f20a-3222-4c3b-9e0b-ec86d22138dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:55.912071 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3f20a-3222-4c3b-9e0b-ec86d22138dc None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:55.912347 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3f20a-3222-4c3b-9e0b-ec86d22138dc None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:55.912619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3f20a-3222-4c3b-9e0b-ec86d22138dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:55.915593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8530bc2-98f1-4a5b-9bf2-40dd2a2212cd None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:55.923538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3f20a-3222-4c3b-9e0b-ec86d22138dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:55.931107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-ecf3f20a-3222-4c3b-9e0b-ec86d22138dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:26:55.931107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:55.931107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:55.931107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:55.931107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:55.931107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:55.931107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:55.931107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:55.931107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:55.931107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:55.931107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:55.931107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:55.931107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:55.931107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:55.931107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:55.931107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:55.932051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:55.932051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:55.932051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:55.932051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:55.932051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:55.932051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:55.932051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:55.932051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:55.932051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:55.932051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:55.932051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:55.932051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:55.932051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:55.932051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:55.932051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:55.932051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:55.932945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:55.932945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:55.932945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:55.932945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:55.932945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:55.932945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:55.932945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:55.932945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:55.932945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:55.932945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:55.932945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:26:55.932945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:55.932945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 511/1045] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:26:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:55.936930 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:55.938175 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3f20a-3222-4c3b-9e0b-ec86d22138dc None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:55.955507 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e48b32-136c-45c1-aafe-e20ada5c743b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:55.956560 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e48b32-136c-45c1-aafe-e20ada5c743b None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:55.957104 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e48b32-136c-45c1-aafe-e20ada5c743b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:55.957676 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e48b32-136c-45c1-aafe-e20ada5c743b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:55.966203 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e48b32-136c-45c1-aafe-e20ada5c743b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:55.968158 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e48b32-136c-45c1-aafe-e20ada5c743b None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:55.982690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-88e48b32-136c-45c1-aafe-e20ada5c743b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:55.982690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:55.982690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:55.982690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:55.982690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:55.982690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:55.982690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:55.982690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:55.982690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:55.982690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:55.982690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:55.982690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:55.982690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:26:55.982690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:26:55.982690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:26:55.983667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:26:55.983667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:55.983667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:55.983667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:55.983667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:55.983667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:55.983667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:55.983667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:55.983667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:55.983667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:55.983667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:55.983667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:55.983667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:55.983667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:55.983667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:55.983667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:55.983667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:55.984584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:26:55.984584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:26:55.984584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:26:55.984584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:26:55.984584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:26:55.984584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:26:55.984584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:26:55.984584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:55.984584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 535/1046] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:55.990029 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e48b32-136c-45c1-aafe-e20ada5c743b None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:56.031069 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:56.044673 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3f20a-3222-4c3b-9e0b-ec86d22138dc None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:56.085603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47886c-2912-487b-8685-67e77e6059d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:56.086266 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47886c-2912-487b-8685-67e77e6059d4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:56.086619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47886c-2912-487b-8685-67e77e6059d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:56.086883 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47886c-2912-487b-8685-67e77e6059d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:56.087609 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47886c-2912-487b-8685-67e77e6059d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:56.087928 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47886c-2912-487b-8685-67e77e6059d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:56.089399 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47886c-2912-487b-8685-67e77e6059d4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:56.095686 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e48b32-136c-45c1-aafe-e20ada5c743b None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:56.102504 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 512/1047] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:26:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:56.103456 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47886c-2912-487b-8685-67e77e6059d4 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:56.139447 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f5531-68db-49e5-9d16-238ec4a08297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:56.140287 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f5531-68db-49e5-9d16-238ec4a08297 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:56.141431 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f5531-68db-49e5-9d16-238ec4a08297 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:56.141715 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f5531-68db-49e5-9d16-238ec4a08297 None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:56.142507 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f5531-68db-49e5-9d16-238ec4a08297 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:56.142848 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f5531-68db-49e5-9d16-238ec4a08297 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:56.144389 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f5531-68db-49e5-9d16-238ec4a08297 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:56.160990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 536/1048] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:26:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:56.166548 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f5531-68db-49e5-9d16-238ec4a08297 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:56.209134 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:56.210658 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47886c-2912-487b-8685-67e77e6059d4 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:56.245772 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ac0d9-942e-44cf-993b-57d4b8e754da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:56.246673 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ac0d9-942e-44cf-993b-57d4b8e754da None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:56.246953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ac0d9-942e-44cf-993b-57d4b8e754da None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:56.247240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ac0d9-942e-44cf-993b-57d4b8e754da None admin] PATH_INFO: `/v3/users/0ee93beaea5542569806d9a3da3c9d2b` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:56.261043 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ac0d9-942e-44cf-993b-57d4b8e754da None admin] RBAC: Authorizing `identity:get_user(user_id=0ee93beaea5542569806d9a3da3c9d2b)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:56.262609 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ac0d9-942e-44cf-993b-57d4b8e754da None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:56.264285 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 513/1049] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:26:56 2022] GET /identity/v3/users/0ee93beaea5542569806d9a3da3c9d2b => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:56.269662 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ac0d9-942e-44cf-993b-57d4b8e754da None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:56.279605 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f5531-68db-49e5-9d16-238ec4a08297 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:56.280630 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:56.317225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e64ae-df0d-4a0c-9bc0-b1301218118d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:56.317875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e64ae-df0d-4a0c-9bc0-b1301218118d None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:56.318160 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e64ae-df0d-4a0c-9bc0-b1301218118d None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:56.318438 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e64ae-df0d-4a0c-9bc0-b1301218118d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:56.319294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e64ae-df0d-4a0c-9bc0-b1301218118d None admin] RBAC: Adding query filter params (role.id=a9a87bede27348b89fa09b9dfd04e89c, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=161c5ed1e47e450290a1f428ee100b68) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:56.319606 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e64ae-df0d-4a0c-9bc0-b1301218118d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:56.321029 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e64ae-df0d-4a0c-9bc0-b1301218118d None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:56.326913 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 537/1050] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:26:56 2022] GET /identity/v3/role_assignments?user.id=161c5ed1e47e450290a1f428ee100b68&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=a9a87bede27348b89fa09b9dfd04e89c => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:56.376251 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ac0d9-942e-44cf-993b-57d4b8e754da None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:56.413984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240b262-4313-471d-aa9e-3ed9106e2854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:56.414596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240b262-4313-471d-aa9e-3ed9106e2854 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:56.414883 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240b262-4313-471d-aa9e-3ed9106e2854 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:56.415159 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240b262-4313-471d-aa9e-3ed9106e2854 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:56.492408 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240b262-4313-471d-aa9e-3ed9106e2854 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:56.493874 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240b262-4313-471d-aa9e-3ed9106e2854 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:56.505628 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:56.596957 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 514/1051] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:26:56 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 11:26:56.602858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e64ae-df0d-4a0c-9bc0-b1301218118d None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:56.702781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e64ae-df0d-4a0c-9bc0-b1301218118d None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:56.704232 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:56.737321 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451d0a3-a631-443a-9906-f96525883b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:56.738183 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451d0a3-a631-443a-9906-f96525883b91 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:56.738465 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451d0a3-a631-443a-9906-f96525883b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:56.738781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451d0a3-a631-443a-9906-f96525883b91 None admin] PATH_INFO: `/v3/projects/f2755c3c84a749d985413b51ff43d333` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:56.739955 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451d0a3-a631-443a-9906-f96525883b91 None admin] RBAC: Authorizing `identity:get_project(project_id=f2755c3c84a749d985413b51ff43d333)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:56.741375 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451d0a3-a631-443a-9906-f96525883b91 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:56.748452 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 538/1052] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:26:56 2022] GET /identity/v3/projects/f2755c3c84a749d985413b51ff43d333 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:56.749650 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240b262-4313-471d-aa9e-3ed9106e2854 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:56.854281 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240b262-4313-471d-aa9e-3ed9106e2854 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:56.863703 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:56.889412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be263ee-77b4-4642-861d-998905f10588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:56.890081 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be263ee-77b4-4642-861d-998905f10588 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:56.890356 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be263ee-77b4-4642-861d-998905f10588 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:56.890657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be263ee-77b4-4642-861d-998905f10588 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:56.891494 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be263ee-77b4-4642-861d-998905f10588 None admin] RBAC: Adding query filter params (role.id=c9e006de0d6c4812ab29d778e9c8a830, scope.project.id=f2755c3c84a749d985413b51ff43d333, user.id=0ee93beaea5542569806d9a3da3c9d2b) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:56.891809 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be263ee-77b4-4642-861d-998905f10588 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:56.893241 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be263ee-77b4-4642-861d-998905f10588 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:56.906953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 515/1053] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:26:56 2022] GET /identity/v3/role_assignments?user.id=0ee93beaea5542569806d9a3da3c9d2b&scope.project.id=f2755c3c84a749d985413b51ff43d333&role.id=c9e006de0d6c4812ab29d778e9c8a830 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:57.479958 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07598427-c979-432c-a14b-19d9e5ade4fa None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:57.480484 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07598427-c979-432c-a14b-19d9e5ade4fa None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:57.480593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07598427-c979-432c-a14b-19d9e5ade4fa None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:57.485029 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 539/1054] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:57.487599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7058af3-a80c-40b7-91b7-b1bf087841d1 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:57.488004 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7058af3-a80c-40b7-91b7-b1bf087841d1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:57.488195 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7058af3-a80c-40b7-91b7-b1bf087841d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:57.506770 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f7058af3-a80c-40b7-91b7-b1bf087841d1 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:57.507954 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f7058af3-a80c-40b7-91b7-b1bf087841d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:57.578461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 516/1055] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:57.601921 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07598427-c979-432c-a14b-19d9e5ade4fa None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:57.704138 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07598427-c979-432c-a14b-19d9e5ade4fa None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:57.737923 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149291d6-c48d-412a-863e-00a5a3a87940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:57.738671 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149291d6-c48d-412a-863e-00a5a3a87940 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:57.738959 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149291d6-c48d-412a-863e-00a5a3a87940 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:57.739238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149291d6-c48d-412a-863e-00a5a3a87940 None admin] PATH_INFO: `/v3/services/image` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:57.739891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149291d6-c48d-412a-863e-00a5a3a87940 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:57.741253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149291d6-c48d-412a-863e-00a5a3a87940 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:57.752908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-149291d6-c48d-412a-863e-00a5a3a87940 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 11:26:57.752908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:57.752908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:26:57.752908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:26:57.752908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:26:57.752908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:26:57.752908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:26:57.752908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:57.752908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 11:26:57.752908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:57.752908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:26:57.752908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:57.752908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:57.752908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:57.752908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:57.752908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:57.752908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:57.753935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:57.753935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:57.753935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:57.753935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:57.753935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:57.753935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:57.753935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:26:57.753935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:26:57.753935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:26:57.753935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:26:57.753935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:57.753935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:57.753935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:57.753935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:57.753935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:57.753935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:57.755175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:57.755175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:57.755175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:57.755175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:57.755175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:57.755175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:57.755175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:57.755175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:57.755175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:57.755175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:57.755175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:26:57.755175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:26:57.755175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 11:26:57.755175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:57.758852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 540/1056] 10.222.0.13 () {56 vars in 1105 bytes} [Thu Jun 23 11:26:57 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:57.760167 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7058af3-a80c-40b7-91b7-b1bf087841d1 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:57.867188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7058af3-a80c-40b7-91b7-b1bf087841d1 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:57.868729 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:57.902733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fc0e2-2188-423e-9117-33d831245240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:57.903414 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fc0e2-2188-423e-9117-33d831245240 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:57.903699 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fc0e2-2188-423e-9117-33d831245240 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:57.904005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fc0e2-2188-423e-9117-33d831245240 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:57.904713 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fc0e2-2188-423e-9117-33d831245240 None admin] RBAC: Adding query filter params (name=image) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:57.905024 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fc0e2-2188-423e-9117-33d831245240 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:57.906396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fc0e2-2188-423e-9117-33d831245240 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:57.916299 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 517/1057] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:57.918444 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149291d6-c48d-412a-863e-00a5a3a87940 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:57.954013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:58.023711 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149291d6-c48d-412a-863e-00a5a3a87940 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:58.059087 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929a454-075c-4dcb-a3d5-e28f304c6ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:58.059751 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929a454-075c-4dcb-a3d5-e28f304c6ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:58.060038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929a454-075c-4dcb-a3d5-e28f304c6ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:58.060316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929a454-075c-4dcb-a3d5-e28f304c6ae1 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:58.061029 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929a454-075c-4dcb-a3d5-e28f304c6ae1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:58.061339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929a454-075c-4dcb-a3d5-e28f304c6ae1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:58.062751 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929a454-075c-4dcb-a3d5-e28f304c6ae1 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:58.072791 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 541/1058] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:58.074352 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fc0e2-2188-423e-9117-33d831245240 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:58.174788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fc0e2-2188-423e-9117-33d831245240 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:58.209338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31158c26-7f68-43f7-a07b-b8adf775e982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:58.209996 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31158c26-7f68-43f7-a07b-b8adf775e982 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:58.210283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31158c26-7f68-43f7-a07b-b8adf775e982 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:58.210583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31158c26-7f68-43f7-a07b-b8adf775e982 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:58.211367 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31158c26-7f68-43f7-a07b-b8adf775e982 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:58.212732 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31158c26-7f68-43f7-a07b-b8adf775e982 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:58.298702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 518/1059] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jun 23 11:26:58 2022] POST /identity/v3/endpoints => generated 327 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:26:58.380873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd48a84-336e-41f8-9c93-ef888b3ab2d6 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:58.381190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd48a84-336e-41f8-9c93-ef888b3ab2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:58.381441 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd48a84-336e-41f8-9c93-ef888b3ab2d6 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:58.383041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 542/1060] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:26:58.389140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1e051-9e1b-4f57-8573-6d5bc7a47a61 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:58.389432 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1e051-9e1b-4f57-8573-6d5bc7a47a61 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:58.389716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1e051-9e1b-4f57-8573-6d5bc7a47a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:58.408550 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-bac1e051-9e1b-4f57-8573-6d5bc7a47a61 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:58.409774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bac1e051-9e1b-4f57-8573-6d5bc7a47a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:58.429282 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:58.494200 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 519/1061] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:26:58.517683 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd48a84-336e-41f8-9c93-ef888b3ab2d6 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:58.616976 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:58.622609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd48a84-336e-41f8-9c93-ef888b3ab2d6 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:58.657146 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98592452-beed-497a-ab68-b619e58f6f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:58.657959 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98592452-beed-497a-ab68-b619e58f6f17 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:58.658241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98592452-beed-497a-ab68-b619e58f6f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:58.658512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98592452-beed-497a-ab68-b619e58f6f17 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:58.668995 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98592452-beed-497a-ab68-b619e58f6f17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:58.677099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-98592452-beed-497a-ab68-b619e58f6f17 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:26:58.677099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:58.677099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:58.677099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:58.677099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:58.677099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:58.677099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:58.677099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:58.677099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:58.677099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:58.677099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:58.677099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:58.677099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:58.677099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:58.677099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:58.677099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:58.678018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:58.678018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:58.678018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:58.678018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:58.678018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:58.678018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:58.678018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:58.678018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:58.678018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:58.678018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:58.678018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:58.678018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:58.678018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:58.678018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:58.678018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:58.678018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:58.679306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:58.679306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:58.679306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:58.679306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:58.679306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:58.679306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:58.679306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:58.679306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:58.679306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:58.679306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:58.679306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:26:58.679306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:58.680429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 543/1062] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Jun 23 11:26:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:58.685125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1e051-9e1b-4f57-8573-6d5bc7a47a61 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:58.788232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1e051-9e1b-4f57-8573-6d5bc7a47a61 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:58.823811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8fff8-9486-4618-a485-bbb6881c91e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:58.824484 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8fff8-9486-4618-a485-bbb6881c91e7 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:58.824766 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8fff8-9486-4618-a485-bbb6881c91e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:58.825036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8fff8-9486-4618-a485-bbb6881c91e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:58.825786 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8fff8-9486-4618-a485-bbb6881c91e7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:58.826094 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8fff8-9486-4618-a485-bbb6881c91e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:58.827483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8fff8-9486-4618-a485-bbb6881c91e7 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:58.841011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 520/1063] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jun 23 11:26:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:58.842313 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98592452-beed-497a-ab68-b619e58f6f17 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:58.918932 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167878f7-1e71-4463-82ab-8f28f3f44e96 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:58.919505 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167878f7-1e71-4463-82ab-8f28f3f44e96 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:58.919961 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167878f7-1e71-4463-82ab-8f28f3f44e96 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:58.921496 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 521/1064] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:26:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:26:58.926525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71bd35-20f8-49e9-9435-411fd4837293 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:58.927042 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71bd35-20f8-49e9-9435-411fd4837293 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:58.927522 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71bd35-20f8-49e9-9435-411fd4837293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:58.944767 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-4e71bd35-20f8-49e9-9435-411fd4837293 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:26:58.945941 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4e71bd35-20f8-49e9-9435-411fd4837293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:58.949873 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:58.953905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98592452-beed-497a-ab68-b619e58f6f17 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:58.989616 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec7838-01b4-4e2c-8b85-80217f20b48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:58.990410 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec7838-01b4-4e2c-8b85-80217f20b48a None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:58.990726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec7838-01b4-4e2c-8b85-80217f20b48a None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:58.991291 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec7838-01b4-4e2c-8b85-80217f20b48a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:59.001692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec7838-01b4-4e2c-8b85-80217f20b48a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:59.003184 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec7838-01b4-4e2c-8b85-80217f20b48a None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:59.020094 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 522/1065] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:26:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:26:59.020621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-0bec7838-01b4-4e2c-8b85-80217f20b48a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:59.020621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:59.020621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:59.020621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:59.020621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:59.020621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:59.020621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:59.020621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:59.020621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:59.020621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:59.020621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:59.020621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:59.020621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:59.020621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:59.020621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:59.021480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:59.021480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:59.021480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:59.021480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:59.021480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:59.021480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:59.021480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:59.021480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:59.021480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:59.021480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:59.021480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:59.021480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:59.021480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:59.021480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:59.021480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:59.021480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:59.021480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:59.022373 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:59.022373 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:59.022373 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:59.022373 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:59.022373 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 544/1066] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:26:59.028742 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71bd35-20f8-49e9-9435-411fd4837293 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:59.047329 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec7838-01b4-4e2c-8b85-80217f20b48a None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:59.061099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:59.137845 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71bd35-20f8-49e9-9435-411fd4837293 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:59.154521 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec7838-01b4-4e2c-8b85-80217f20b48a None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:59.173878 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ecd35-b5f5-4f57-b57a-e533bb8a0801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:59.174470 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ecd35-b5f5-4f57-b57a-e533bb8a0801 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:59.174771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ecd35-b5f5-4f57-b57a-e533bb8a0801 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:59.175050 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ecd35-b5f5-4f57-b57a-e533bb8a0801 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:59.175762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ecd35-b5f5-4f57-b57a-e533bb8a0801 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:59.176071 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ecd35-b5f5-4f57-b57a-e533bb8a0801 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:59.177426 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ecd35-b5f5-4f57-b57a-e533bb8a0801 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:59.186013 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:59.189521 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 523/1067] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:59.191054 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87829b2b-fec5-48cb-b557-69809bc115db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:59.192208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87829b2b-fec5-48cb-b557-69809bc115db None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:59.194420 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87829b2b-fec5-48cb-b557-69809bc115db None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:59.194420 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87829b2b-fec5-48cb-b557-69809bc115db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:59.195582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ecd35-b5f5-4f57-b57a-e533bb8a0801 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:59.204109 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87829b2b-fec5-48cb-b557-69809bc115db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:26:59.211740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-87829b2b-fec5-48cb-b557-69809bc115db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:26:59.211740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:59.211740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:59.211740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:59.211740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:59.211740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:59.211740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:59.211740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:59.211740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:59.211740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:59.211740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:59.211740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:59.211740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:26:59.211740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:26:59.211740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:26:59.211740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:26:59.212928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:26:59.212928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:26:59.212928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:26:59.212928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:26:59.212928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:59.212928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:59.212928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:59.212928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:59.212928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:59.212928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:59.212928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:59.212928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:59.212928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:59.212928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:59.212928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:59.212928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:59.215124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:59.215124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:59.215124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:59.215124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:59.215124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:26:59.215124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:26:59.215124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:26:59.215124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:26:59.215124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:26:59.215124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:26:59.215124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:26:59.215124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:26:59.218704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 545/1068] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:26:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:59.219523 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87829b2b-fec5-48cb-b557-69809bc115db None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:59.291573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:59.305985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ecd35-b5f5-4f57-b57a-e533bb8a0801 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:59.322743 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87829b2b-fec5-48cb-b557-69809bc115db None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:59.345414 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9acd932-02bd-4504-ac92-c40827948d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:59.346476 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acd932-02bd-4504-ac92-c40827948d8d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:59.346928 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acd932-02bd-4504-ac92-c40827948d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:59.347220 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acd932-02bd-4504-ac92-c40827948d8d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:59.355737 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9acd932-02bd-4504-ac92-c40827948d8d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:59.357203 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9acd932-02bd-4504-ac92-c40827948d8d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:59.358388 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063cfc11-2f40-4e6a-a32a-ced583e69c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:59.359082 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cfc11-2f40-4e6a-a32a-ced583e69c1e None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:59.359367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cfc11-2f40-4e6a-a32a-ced583e69c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:59.359979 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cfc11-2f40-4e6a-a32a-ced583e69c1e None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:59.360378 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063cfc11-2f40-4e6a-a32a-ced583e69c1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:59.360686 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063cfc11-2f40-4e6a-a32a-ced583e69c1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:59.362039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063cfc11-2f40-4e6a-a32a-ced583e69c1e None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:59.371199 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 546/1069] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:26:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:59.372253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-c9acd932-02bd-4504-ac92-c40827948d8d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:26:59.372253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:59.372253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:59.372253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:59.372253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:59.372253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:59.372253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:59.372253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:59.372253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:59.372253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:59.372253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:59.372253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:59.372253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:26:59.372253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:26:59.372253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:26:59.372253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:26:59.373149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:59.373149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:59.373149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:26:59.373149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:59.373149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:26:59.373149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:26:59.373149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:59.373149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:59.373149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:59.373149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:59.373149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:59.373149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:59.373149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:59.373149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:59.373149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:59.373149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:59.373149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:59.373149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:59.374098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:59.374098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:59.374098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:26:59.374098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:26:59.374098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:26:59.374098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:26:59.374098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:26:59.374098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:26:59.374098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:26:59.374098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:59.376597 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063cfc11-2f40-4e6a-a32a-ced583e69c1e None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:59.378219 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 524/1070] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jun 23 11:26:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:59.379008 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9acd932-02bd-4504-ac92-c40827948d8d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:59.443820 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:59.489634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063cfc11-2f40-4e6a-a32a-ced583e69c1e None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:59.492581 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9acd932-02bd-4504-ac92-c40827948d8d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:59.522096 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:59.529322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e133ef-5a09-42e5-b292-8ec22c56cdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:59.532484 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655fb3c-e4c1-4cb9-85c7-78afc8d7c3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:59.532878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e133ef-5a09-42e5-b292-8ec22c56cdd3 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:59.532878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e133ef-5a09-42e5-b292-8ec22c56cdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:59.532878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e133ef-5a09-42e5-b292-8ec22c56cdd3 None admin] PATH_INFO: `/v3/users/0ee93beaea5542569806d9a3da3c9d2b` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:59.533066 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655fb3c-e4c1-4cb9-85c7-78afc8d7c3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:59.533066 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655fb3c-e4c1-4cb9-85c7-78afc8d7c3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:59.533066 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655fb3c-e4c1-4cb9-85c7-78afc8d7c3f2 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:59.533258 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e133ef-5a09-42e5-b292-8ec22c56cdd3 None admin] RBAC: Authorizing `identity:get_user(user_id=0ee93beaea5542569806d9a3da3c9d2b)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:59.533326 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655fb3c-e4c1-4cb9-85c7-78afc8d7c3f2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:59.533326 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655fb3c-e4c1-4cb9-85c7-78afc8d7c3f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:59.533457 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e133ef-5a09-42e5-b292-8ec22c56cdd3 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:59.534034 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655fb3c-e4c1-4cb9-85c7-78afc8d7c3f2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:59.535151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 547/1071] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:26:59 2022] GET /identity/v3/users/0ee93beaea5542569806d9a3da3c9d2b => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:59.540509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e133ef-5a09-42e5-b292-8ec22c56cdd3 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:59.554349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 525/1072] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jun 23 11:26:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:26:59.559756 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655fb3c-e4c1-4cb9-85c7-78afc8d7c3f2 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:59.651909 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e133ef-5a09-42e5-b292-8ec22c56cdd3 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:59.661367 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655fb3c-e4c1-4cb9-85c7-78afc8d7c3f2 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:59.679133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:59.687245 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d6423-d59b-4e05-8a64-eea36002be58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:59.692569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d6423-d59b-4e05-8a64-eea36002be58 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:59.692569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d6423-d59b-4e05-8a64-eea36002be58 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:59.692569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d6423-d59b-4e05-8a64-eea36002be58 None admin] PATH_INFO: `/v3/projects/f2755c3c84a749d985413b51ff43d333` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:59.692569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d6423-d59b-4e05-8a64-eea36002be58 None admin] RBAC: Authorizing `identity:get_project(project_id=f2755c3c84a749d985413b51ff43d333)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:59.692569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d6423-d59b-4e05-8a64-eea36002be58 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:59.693480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 548/1073] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:26:59 2022] GET /identity/v3/projects/f2755c3c84a749d985413b51ff43d333 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:59.696547 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c95b58-b9ec-455a-9430-3ebdaa11e5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:59.697280 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c95b58-b9ec-455a-9430-3ebdaa11e5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:59.697562 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c95b58-b9ec-455a-9430-3ebdaa11e5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:59.698625 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c95b58-b9ec-455a-9430-3ebdaa11e5b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:59.699661 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d6423-d59b-4e05-8a64-eea36002be58 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:59.708750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c95b58-b9ec-455a-9430-3ebdaa11e5b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:59.710185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c95b58-b9ec-455a-9430-3ebdaa11e5b8 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:59.729320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-79c95b58-b9ec-455a-9430-3ebdaa11e5b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:59.729320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:26:59.729320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:26:59.729320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:26:59.729320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:26:59.729320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:26:59.729320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:26:59.729320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:26:59.729320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:26:59.729320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:26:59.729320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:26:59.729320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:26:59.729320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:26:59.729320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:26:59.729320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:26:59.730418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:26:59.730418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:26:59.730418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:26:59.730418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:26:59.730418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:26:59.730418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:26:59.730418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:26:59.730418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:26:59.730418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:26:59.730418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:26:59.730418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:26:59.730418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:26:59.730418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:26:59.730418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:26:59.730418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:26:59.730418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:26:59.730418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:26:59.731498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:26:59.731498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:26:59.731498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:26:59.731498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:26:59.731498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 526/1074] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:26:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:26:59.736063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c95b58-b9ec-455a-9430-3ebdaa11e5b8 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:59.773422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:59.809583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d6423-d59b-4e05-8a64-eea36002be58 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:59.836784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c95b58-b9ec-455a-9430-3ebdaa11e5b8 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:59.849327 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a42541-3dbf-45de-9d57-4089605aee1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:59.850215 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a42541-3dbf-45de-9d57-4089605aee1b None admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:59.850501 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a42541-3dbf-45de-9d57-4089605aee1b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:59.850806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a42541-3dbf-45de-9d57-4089605aee1b None admin] PATH_INFO: `/v3/projects/f2755c3c84a749d985413b51ff43d333/users/0ee93beaea5542569806d9a3da3c9d2b/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:59.853169 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a42541-3dbf-45de-9d57-4089605aee1b None admin] RBAC: Authorizing `identity:create_grant(project_id=f2755c3c84a749d985413b51ff43d333, user_id=0ee93beaea5542569806d9a3da3c9d2b, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:59.854843 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a42541-3dbf-45de-9d57-4089605aee1b None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:59.872205 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d3390-8c5b-4b26-8adc-470853c945cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:59.872873 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d3390-8c5b-4b26-8adc-470853c945cc None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:59.873154 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d3390-8c5b-4b26-8adc-470853c945cc None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:59.873427 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d3390-8c5b-4b26-8adc-470853c945cc None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:59.874144 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d3390-8c5b-4b26-8adc-470853c945cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:26:59.874451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d3390-8c5b-4b26-8adc-470853c945cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:59.875847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d3390-8c5b-4b26-8adc-470853c945cc None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:59.887656 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 527/1075] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:26:59.893090 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d3390-8c5b-4b26-8adc-470853c945cc None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:59.923363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:59.957739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 549/1076] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:26:59 2022] PUT /identity/v3/projects/f2755c3c84a749d985413b51ff43d333/users/0ee93beaea5542569806d9a3da3c9d2b/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:27:00.000954 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d3390-8c5b-4b26-8adc-470853c945cc None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:00.040370 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8295f3d7-7946-495d-aeb4-a6b795a1bead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:00.041186 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295f3d7-7946-495d-aeb4-a6b795a1bead None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:00.041522 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295f3d7-7946-495d-aeb4-a6b795a1bead None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:00.042361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295f3d7-7946-495d-aeb4-a6b795a1bead None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:00.050110 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8295f3d7-7946-495d-aeb4-a6b795a1bead None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:00.051600 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8295f3d7-7946-495d-aeb4-a6b795a1bead None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:00.068216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-8295f3d7-7946-495d-aeb4-a6b795a1bead None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:27:00.068216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:27:00.068216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:27:00.068216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:27:00.068216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:27:00.068216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:27:00.068216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:27:00.068216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:27:00.068216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:27:00.068216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:27:00.068216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:27:00.068216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:27:00.068216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:27:00.068216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:27:00.068216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:27:00.069351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:27:00.069351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:27:00.069351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:27:00.069351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:27:00.069351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:27:00.069351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:27:00.069351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:27:00.069351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:27:00.069351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:27:00.069351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:27:00.069351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:27:00.069351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:27:00.069351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:27:00.069351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:27:00.069351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:27:00.069351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:27:00.069351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:27:00.075659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:27:00.075659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:27:00.075659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:27:00.075659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:27:00.075659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:27:00.075659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:27:00.075659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:27:00.075659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:27:00.075659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 528/1077] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:26:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:27:00.078344 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a42541-3dbf-45de-9d57-4089605aee1b None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:00.189957 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a42541-3dbf-45de-9d57-4089605aee1b None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:00.217103 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:00.225281 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b43575-25e1-4bc0-8ef5-6f2d7f6148af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:00.225945 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b43575-25e1-4bc0-8ef5-6f2d7f6148af None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:00.227123 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b43575-25e1-4bc0-8ef5-6f2d7f6148af None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:00.227123 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b43575-25e1-4bc0-8ef5-6f2d7f6148af None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:00.227294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b43575-25e1-4bc0-8ef5-6f2d7f6148af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:00.227614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b43575-25e1-4bc0-8ef5-6f2d7f6148af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:00.229226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b43575-25e1-4bc0-8ef5-6f2d7f6148af None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:00.241345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 550/1078] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:27:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:00.247596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8295f3d7-7946-495d-aeb4-a6b795a1bead None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:00.346603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:00.356867 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8295f3d7-7946-495d-aeb4-a6b795a1bead None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:00.392054 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94733a27-7c74-4eee-b3a8-5e577aedc9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:00.392946 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94733a27-7c74-4eee-b3a8-5e577aedc9bc None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:00.393229 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94733a27-7c74-4eee-b3a8-5e577aedc9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:00.393504 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94733a27-7c74-4eee-b3a8-5e577aedc9bc None admin] PATH_INFO: `/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/161c5ed1e47e450290a1f428ee100b68/roles/a9a87bede27348b89fa09b9dfd04e89c` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:00.395779 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94733a27-7c74-4eee-b3a8-5e577aedc9bc None admin] RBAC: Authorizing `identity:create_grant(project_id=886c2939a14b48febc2e2fe639c8f1e4, user_id=161c5ed1e47e450290a1f428ee100b68, role_id=a9a87bede27348b89fa09b9dfd04e89c)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:00.397219 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94733a27-7c74-4eee-b3a8-5e577aedc9bc None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:00.403677 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539e370-6b54-4172-b62d-3025a6619231 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:00.403973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539e370-6b54-4172-b62d-3025a6619231 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:00.404255 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539e370-6b54-4172-b62d-3025a6619231 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:00.405635 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 551/1079] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:27:00.410638 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4aaf65-3284-4870-965d-34a9647be191 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:00.410933 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4aaf65-3284-4870-965d-34a9647be191 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:00.415505 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4aaf65-3284-4870-965d-34a9647be191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:00.428353 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-4f4aaf65-3284-4870-965d-34a9647be191 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:27:00.429526 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-4f4aaf65-3284-4870-965d-34a9647be191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:00.478773 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 529/1080] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:27:00 2022] PUT /identity/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/161c5ed1e47e450290a1f428ee100b68/roles/a9a87bede27348b89fa09b9dfd04e89c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:27:00.512002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 552/1081] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:27:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:27:00.535750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94733a27-7c74-4eee-b3a8-5e577aedc9bc None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:00.653185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94733a27-7c74-4eee-b3a8-5e577aedc9bc None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:00.689801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f005b99-5006-4944-8873-0b1f14bbacc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:00.690458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f005b99-5006-4944-8873-0b1f14bbacc8 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:00.690785 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f005b99-5006-4944-8873-0b1f14bbacc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:00.691060 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f005b99-5006-4944-8873-0b1f14bbacc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:00.711975 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:00.762971 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f005b99-5006-4944-8873-0b1f14bbacc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:00.764391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f005b99-5006-4944-8873-0b1f14bbacc8 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:00.871328 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 530/1082] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:27:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:27:00.872946 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4aaf65-3284-4870-965d-34a9647be191 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:00.974869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4aaf65-3284-4870-965d-34a9647be191 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:00.993045 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:01.010948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a2ac1-a96b-4f4d-af8f-e413ec782f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:01.011750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a2ac1-a96b-4f4d-af8f-e413ec782f1b None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:01.012034 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a2ac1-a96b-4f4d-af8f-e413ec782f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:01.012310 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a2ac1-a96b-4f4d-af8f-e413ec782f1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:01.022515 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a2ac1-a96b-4f4d-af8f-e413ec782f1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:01.023974 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a2ac1-a96b-4f4d-af8f-e413ec782f1b None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:01.043503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-105a2ac1-a96b-4f4d-af8f-e413ec782f1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:27:01.043503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:27:01.043503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:27:01.043503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:27:01.043503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:27:01.043503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:27:01.043503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:27:01.043503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:27:01.043503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:27:01.043503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:27:01.043503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:27:01.043503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:27:01.043503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:27:01.043503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:27:01.043503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:27:01.044462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:27:01.044462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:27:01.044462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:27:01.044462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:27:01.044462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:27:01.044462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:27:01.044462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:27:01.044462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:27:01.044462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:27:01.044462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:27:01.044462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:27:01.044462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:27:01.044462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:27:01.044462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:27:01.044462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:27:01.044462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:27:01.044462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:27:01.045377 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:27:01.045377 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:27:01.045377 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:27:01.045377 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:27:01.046146 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 553/1083] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:27:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:27:01.050754 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f005b99-5006-4944-8873-0b1f14bbacc8 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:01.104133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:01.153578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f005b99-5006-4944-8873-0b1f14bbacc8 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:01.189648 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19d51f-ec4b-42eb-b119-9fbbc68bc3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:01.190313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19d51f-ec4b-42eb-b119-9fbbc68bc3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:01.190620 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19d51f-ec4b-42eb-b119-9fbbc68bc3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:01.190904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19d51f-ec4b-42eb-b119-9fbbc68bc3d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:01.191625 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19d51f-ec4b-42eb-b119-9fbbc68bc3d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:01.192184 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19d51f-ec4b-42eb-b119-9fbbc68bc3d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:01.193555 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19d51f-ec4b-42eb-b119-9fbbc68bc3d0 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:01.210385 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 531/1084] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:27:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:27:02.013476 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60064b9-0b90-4eb9-975d-8dca7607f2f5 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:02.014003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60064b9-0b90-4eb9-975d-8dca7607f2f5 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:02.014003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60064b9-0b90-4eb9-975d-8dca7607f2f5 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:02.015532 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 554/1085] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:27:02.020725 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6809d1-0621-4b38-b1a2-9c06912a3455 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:02.021134 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6809d1-0621-4b38-b1a2-9c06912a3455 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:02.021293 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6809d1-0621-4b38-b1a2-9c06912a3455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:02.038487 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-fb6809d1-0621-4b38-b1a2-9c06912a3455 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:27:02.039684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-fb6809d1-0621-4b38-b1a2-9c06912a3455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:02.105426 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 532/1086] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:27:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:27:02.128797 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60064b9-0b90-4eb9-975d-8dca7607f2f5 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:02.229343 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60064b9-0b90-4eb9-975d-8dca7607f2f5 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:02.263125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34e110-9fbf-4dbc-977f-f36bb77aa9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:02.263989 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34e110-9fbf-4dbc-977f-f36bb77aa9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:02.264218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34e110-9fbf-4dbc-977f-f36bb77aa9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:02.264490 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34e110-9fbf-4dbc-977f-f36bb77aa9c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:02.274364 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34e110-9fbf-4dbc-977f-f36bb77aa9c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:27:02.282878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-6b34e110-9fbf-4dbc-977f-f36bb77aa9c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:27:02.282878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:27:02.282878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:27:02.282878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:27:02.282878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:27:02.282878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:27:02.282878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:27:02.282878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:27:02.282878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:27:02.282878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:27:02.282878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:27:02.282878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:27:02.282878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:27:02.282878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:27:02.282878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:27:02.282878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:27:02.284016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:27:02.284016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:27:02.284016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:27:02.284016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:27:02.284016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:27:02.284016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:27:02.284016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:27:02.284016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:27:02.284016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:27:02.284016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:27:02.284016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:27:02.284016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:27:02.284016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:27:02.284016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:27:02.284016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:27:02.284016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:27:02.285276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:27:02.285276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:27:02.285276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:27:02.285276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:27:02.285276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:27:02.285276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:27:02.285276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:27:02.285276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:27:02.285276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:27:02.285276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:27:02.285276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:27:02.285276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:27:02.286692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 555/1087] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:27:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:27:02.289901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6809d1-0621-4b38-b1a2-9c06912a3455 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:02.332238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:02.389194 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6809d1-0621-4b38-b1a2-9c06912a3455 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:02.424617 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da75b4-175f-4834-8b90-fc00e48d81dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:02.425291 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da75b4-175f-4834-8b90-fc00e48d81dd None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:02.425578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da75b4-175f-4834-8b90-fc00e48d81dd None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:02.425853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da75b4-175f-4834-8b90-fc00e48d81dd None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:02.426865 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da75b4-175f-4834-8b90-fc00e48d81dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:02.426989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da75b4-175f-4834-8b90-fc00e48d81dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:02.428431 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da75b4-175f-4834-8b90-fc00e48d81dd None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:02.439578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 533/1088] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:27:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:02.446275 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34e110-9fbf-4dbc-977f-f36bb77aa9c3 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:02.481261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:02.548621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a4a89-dbe3-49f4-8beb-3c261b106311 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:02.548817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a4a89-dbe3-49f4-8beb-3c261b106311 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:02.549094 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a4a89-dbe3-49f4-8beb-3c261b106311 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:02.550733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 534/1089] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:27:02.551042 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34e110-9fbf-4dbc-977f-f36bb77aa9c3 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:02.555307 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2692c-9c02-44cc-93d2-af0ffde948fd None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:02.555594 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2692c-9c02-44cc-93d2-af0ffde948fd None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:02.555877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2692c-9c02-44cc-93d2-af0ffde948fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:02.573133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c1d2692c-9c02-44cc-93d2-af0ffde948fd None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:27:02.574319 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c1d2692c-9c02-44cc-93d2-af0ffde948fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:02.584842 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d6549-84dc-4d89-a0dc-14709d9e8bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:02.585707 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d6549-84dc-4d89-a0dc-14709d9e8bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:02.585988 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d6549-84dc-4d89-a0dc-14709d9e8bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:02.586264 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d6549-84dc-4d89-a0dc-14709d9e8bf9 None admin] PATH_INFO: `/v3/users/0ee93beaea5542569806d9a3da3c9d2b` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:02.587469 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d6549-84dc-4d89-a0dc-14709d9e8bf9 None admin] RBAC: Authorizing `identity:get_user(user_id=0ee93beaea5542569806d9a3da3c9d2b)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:02.589414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d6549-84dc-4d89-a0dc-14709d9e8bf9 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:02.591055 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 556/1090] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:27:02 2022] GET /identity/v3/users/0ee93beaea5542569806d9a3da3c9d2b => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:02.596773 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d6549-84dc-4d89-a0dc-14709d9e8bf9 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:02.644823 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 535/1091] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:27:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:27:02.668732 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2692c-9c02-44cc-93d2-af0ffde948fd None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:02.690203 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:02.694213 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d6549-84dc-4d89-a0dc-14709d9e8bf9 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:02.698928 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:02.727924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888033d-a9fe-41d6-b0e4-6441c6652b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:02.728564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888033d-a9fe-41d6-b0e4-6441c6652b64 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:02.728862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888033d-a9fe-41d6-b0e4-6441c6652b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:02.729629 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888033d-a9fe-41d6-b0e4-6441c6652b64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:02.765172 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2692c-9c02-44cc-93d2-af0ffde948fd None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:02.797436 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888033d-a9fe-41d6-b0e4-6441c6652b64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:02.798975 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888033d-a9fe-41d6-b0e4-6441c6652b64 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:02.801558 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e290e-9873-49d1-a89e-cf649ea0c8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:02.802365 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e290e-9873-49d1-a89e-cf649ea0c8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:02.802683 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e290e-9873-49d1-a89e-cf649ea0c8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:02.802959 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e290e-9873-49d1-a89e-cf649ea0c8e0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:02.813264 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e290e-9873-49d1-a89e-cf649ea0c8e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:27:02.821645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-d16e290e-9873-49d1-a89e-cf649ea0c8e0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:27:02.821645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:27:02.821645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:27:02.821645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:27:02.821645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:27:02.821645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:27:02.821645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:27:02.821645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:27:02.821645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:27:02.821645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:27:02.821645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:27:02.821645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:27:02.821645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:27:02.821645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:27:02.821645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:27:02.821645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:27:02.823882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:27:02.823882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:27:02.823882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:27:02.823882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:27:02.823882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:27:02.823882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:27:02.823882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:27:02.823882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:27:02.823882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:27:02.823882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:27:02.823882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:27:02.823882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:27:02.823882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:27:02.823882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:27:02.823882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:27:02.823882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:27:02.825402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:27:02.825402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:27:02.825402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:27:02.825402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:27:02.825402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:27:02.825402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:27:02.825402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:27:02.825402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:27:02.825402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:27:02.825402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:27:02.825402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:27:02.825402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:27:02.825402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 536/1092] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Jun 23 11:27:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:27:02.829306 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e290e-9873-49d1-a89e-cf649ea0c8e0 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:02.898280 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:02.903171 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 557/1093] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:27:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 11:27:02.908984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888033d-a9fe-41d6-b0e4-6441c6652b64 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:02.923211 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:02.931424 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e290e-9873-49d1-a89e-cf649ea0c8e0 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:02.966822 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cff72-32e8-446a-87d1-c7ac0017bd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:02.967483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cff72-32e8-446a-87d1-c7ac0017bd41 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:02.967763 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cff72-32e8-446a-87d1-c7ac0017bd41 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:02.968037 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cff72-32e8-446a-87d1-c7ac0017bd41 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:02.968782 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69cff72-32e8-446a-87d1-c7ac0017bd41 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:02.969101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69cff72-32e8-446a-87d1-c7ac0017bd41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:02.970460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69cff72-32e8-446a-87d1-c7ac0017bd41 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:02.978958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 537/1094] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jun 23 11:27:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:27:02.984509 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cff72-32e8-446a-87d1-c7ac0017bd41 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:03.005705 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888033d-a9fe-41d6-b0e4-6441c6652b64 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:03.039826 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d468c3-dc74-44fd-a4d8-eec0f73ada91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:03.040674 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d468c3-dc74-44fd-a4d8-eec0f73ada91 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:03.040961 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d468c3-dc74-44fd-a4d8-eec0f73ada91 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:03.041253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d468c3-dc74-44fd-a4d8-eec0f73ada91 None admin] PATH_INFO: `/v3/projects/f2755c3c84a749d985413b51ff43d333` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:03.042588 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d468c3-dc74-44fd-a4d8-eec0f73ada91 None admin] RBAC: Authorizing `identity:get_project(project_id=f2755c3c84a749d985413b51ff43d333)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:03.044020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d468c3-dc74-44fd-a4d8-eec0f73ada91 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:03.045701 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 558/1095] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:27:02 2022] GET /identity/v3/projects/f2755c3c84a749d985413b51ff43d333 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:27:03.051591 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d468c3-dc74-44fd-a4d8-eec0f73ada91 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:03.082451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cff72-32e8-446a-87d1-c7ac0017bd41 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:03.113995 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:03.114873 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c1c47-55c8-4bff-9cb1-b61ee1121959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:03.115633 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c1c47-55c8-4bff-9cb1-b61ee1121959 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:03.115911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c1c47-55c8-4bff-9cb1-b61ee1121959 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:03.116192 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c1c47-55c8-4bff-9cb1-b61ee1121959 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:03.127061 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c1c47-55c8-4bff-9cb1-b61ee1121959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:03.128502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c1c47-55c8-4bff-9cb1-b61ee1121959 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:03.147435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-0b3c1c47-55c8-4bff-9cb1-b61ee1121959 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:27:03.147435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:27:03.147435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:27:03.147435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:27:03.147435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:27:03.147435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:27:03.147435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:27:03.147435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:27:03.147435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:27:03.147435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:27:03.147435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:27:03.147435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:27:03.147435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:27:03.147435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:27:03.147435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:27:03.148363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:27:03.148363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:27:03.148363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:27:03.148363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:27:03.148363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:27:03.148363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:27:03.148363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:27:03.148363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:27:03.148363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:27:03.148363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:27:03.148363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:27:03.148363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:27:03.148363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:27:03.148363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:27:03.148363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:27:03.148363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:27:03.148363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:27:03.149274 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:27:03.149274 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:27:03.149274 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:27:03.149274 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:27:03.149274 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 538/1096] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:27:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:27:03.152545 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d468c3-dc74-44fd-a4d8-eec0f73ada91 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:03.154440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c1c47-55c8-4bff-9cb1-b61ee1121959 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:03.163493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:03.186212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7eab46-30d6-4490-9585-05af1b0c262a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:03.186954 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7eab46-30d6-4490-9585-05af1b0c262a None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:03.187238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7eab46-30d6-4490-9585-05af1b0c262a None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:03.187514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7eab46-30d6-4490-9585-05af1b0c262a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:03.188333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7eab46-30d6-4490-9585-05af1b0c262a None admin] RBAC: Adding query filter params (role.id=c9e006de0d6c4812ab29d778e9c8a830, scope.project.id=f2755c3c84a749d985413b51ff43d333, user.id=0ee93beaea5542569806d9a3da3c9d2b) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:03.188647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7eab46-30d6-4490-9585-05af1b0c262a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:03.190072 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7eab46-30d6-4490-9585-05af1b0c262a None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:03.196336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 559/1097] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:27:03 2022] GET /identity/v3/role_assignments?user.id=0ee93beaea5542569806d9a3da3c9d2b&scope.project.id=f2755c3c84a749d985413b51ff43d333&role.id=c9e006de0d6c4812ab29d778e9c8a830 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:27:03.261276 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c1c47-55c8-4bff-9cb1-b61ee1121959 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:03.300923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2606aa-928a-43c4-8cab-f3c5b3b0e09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:03.301664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2606aa-928a-43c4-8cab-f3c5b3b0e09f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:03.301890 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2606aa-928a-43c4-8cab-f3c5b3b0e09f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:03.302166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2606aa-928a-43c4-8cab-f3c5b3b0e09f None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:03.302941 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2606aa-928a-43c4-8cab-f3c5b3b0e09f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:03.303251 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2606aa-928a-43c4-8cab-f3c5b3b0e09f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:03.304655 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2606aa-928a-43c4-8cab-f3c5b3b0e09f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:03.319163 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 539/1098] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:27:03.325552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7eab46-30d6-4490-9585-05af1b0c262a None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:03.426684 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7eab46-30d6-4490-9585-05af1b0c262a None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:03.454088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:03.459576 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c0f98-b5b4-43d2-8e2c-178d78ac3eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:03.460362 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c0f98-b5b4-43d2-8e2c-178d78ac3eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:03.464960 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c0f98-b5b4-43d2-8e2c-178d78ac3eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:03.464960 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c0f98-b5b4-43d2-8e2c-178d78ac3eb1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:03.468548 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c0f98-b5b4-43d2-8e2c-178d78ac3eb1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:03.470035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c0f98-b5b4-43d2-8e2c-178d78ac3eb1 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:03.484864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-af8c0f98-b5b4-43d2-8e2c-178d78ac3eb1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:27:03.484864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:27:03.484864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:27:03.484864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:27:03.484864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:27:03.484864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:27:03.484864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:27:03.484864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:27:03.484864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:27:03.484864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:27:03.484864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:27:03.484864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:27:03.484864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:27:03.484864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:27:03.484864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:27:03.484864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:27:03.485881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:27:03.485881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:27:03.485881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:27:03.485881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:27:03.485881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:27:03.485881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:27:03.485881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:27:03.485881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:27:03.485881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:27:03.485881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:27:03.485881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:27:03.485881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:27:03.485881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:27:03.485881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:27:03.485881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:27:03.485881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:27:03.485881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:27:03.485881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:27:03.487180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:27:03.487180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:27:03.487180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:27:03.487180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:27:03.487180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:27:03.487180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:27:03.487180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:27:03.487180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:27:03.487180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:27:03.487180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:27:03.495168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 560/1099] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jun 23 11:27:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:27:03.496857 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2606aa-928a-43c4-8cab-f3c5b3b0e09f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:03.562510 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:03.596553 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2606aa-928a-43c4-8cab-f3c5b3b0e09f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:03.630475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd85925-6a33-4cb7-a777-e5e79cac5abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:03.631161 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd85925-6a33-4cb7-a777-e5e79cac5abf None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:03.631439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd85925-6a33-4cb7-a777-e5e79cac5abf None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:03.631710 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd85925-6a33-4cb7-a777-e5e79cac5abf None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:03.632538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd85925-6a33-4cb7-a777-e5e79cac5abf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:03.632844 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd85925-6a33-4cb7-a777-e5e79cac5abf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:03.634281 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd85925-6a33-4cb7-a777-e5e79cac5abf None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:03.652123 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 540/1100] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jun 23 11:27:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:03.657700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c0f98-b5b4-43d2-8e2c-178d78ac3eb1 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:03.754963 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c0f98-b5b4-43d2-8e2c-178d78ac3eb1 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:03.786940 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df0f5c-abeb-4157-af81-439f742b0905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:03.787581 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df0f5c-abeb-4157-af81-439f742b0905 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:03.787861 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df0f5c-abeb-4157-af81-439f742b0905 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:03.788135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df0f5c-abeb-4157-af81-439f742b0905 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:03.843446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:03.853406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df0f5c-abeb-4157-af81-439f742b0905 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:03.855580 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df0f5c-abeb-4157-af81-439f742b0905 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:03.958944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 561/1101] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:27:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:27:03.965201 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd85925-6a33-4cb7-a777-e5e79cac5abf None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:04.062974 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd85925-6a33-4cb7-a777-e5e79cac5abf None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:04.085904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:04.096830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf40d5c-bb6c-45b2-bf38-a7f4e2bf4993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:04.097555 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf40d5c-bb6c-45b2-bf38-a7f4e2bf4993 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:04.097840 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf40d5c-bb6c-45b2-bf38-a7f4e2bf4993 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:04.098114 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf40d5c-bb6c-45b2-bf38-a7f4e2bf4993 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:04.108400 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf40d5c-bb6c-45b2-bf38-a7f4e2bf4993 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:04.109860 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf40d5c-bb6c-45b2-bf38-a7f4e2bf4993 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:04.127155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-6cf40d5c-bb6c-45b2-bf38-a7f4e2bf4993 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:27:04.127155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:27:04.127155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:27:04.127155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:27:04.127155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:27:04.127155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:27:04.127155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:27:04.127155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:27:04.127155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:27:04.127155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:27:04.127155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:27:04.127155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:27:04.127155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:27:04.127155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:27:04.127155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:27:04.128213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:27:04.128213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:27:04.128213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:27:04.128213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:27:04.128213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:27:04.128213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:27:04.128213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:27:04.128213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:27:04.128213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:27:04.128213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:27:04.128213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:27:04.128213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:27:04.128213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:27:04.128213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:27:04.128213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:27:04.128213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:27:04.128213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:27:04.129134 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:27:04.129134 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:27:04.129134 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:27:04.129134 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:27:04.129134 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 541/1102] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:27:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:27:04.133928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df0f5c-abeb-4157-af81-439f742b0905 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:04.223261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:04.232808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df0f5c-abeb-4157-af81-439f742b0905 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:04.266303 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca8f2b-16e8-4b97-b6a1-096cba4b2bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:04.267059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca8f2b-16e8-4b97-b6a1-096cba4b2bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:04.267600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca8f2b-16e8-4b97-b6a1-096cba4b2bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:04.268101 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca8f2b-16e8-4b97-b6a1-096cba4b2bb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:04.269250 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca8f2b-16e8-4b97-b6a1-096cba4b2bb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:04.269555 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca8f2b-16e8-4b97-b6a1-096cba4b2bb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:04.270951 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca8f2b-16e8-4b97-b6a1-096cba4b2bb4 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:04.283282 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 562/1103] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:04.288800 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf40d5c-bb6c-45b2-bf38-a7f4e2bf4993 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:04.389722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf40d5c-bb6c-45b2-bf38-a7f4e2bf4993 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:04.423156 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecab9ec-5f81-48d7-b750-38e0f1f52c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:04.423942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecab9ec-5f81-48d7-b750-38e0f1f52c2a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:04.424222 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecab9ec-5f81-48d7-b750-38e0f1f52c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:04.424491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecab9ec-5f81-48d7-b750-38e0f1f52c2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:04.430941 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecab9ec-5f81-48d7-b750-38e0f1f52c2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:04.432422 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecab9ec-5f81-48d7-b750-38e0f1f52c2a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:04.446104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-0ecab9ec-5f81-48d7-b750-38e0f1f52c2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:27:04.446104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:27:04.446104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:27:04.446104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:27:04.446104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:27:04.446104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:27:04.446104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:27:04.446104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:27:04.446104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:27:04.446104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:27:04.446104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:27:04.446104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:27:04.446104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:27:04.446104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:27:04.446104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:27:04.447193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:27:04.447193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:27:04.447193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:27:04.447193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:27:04.447193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:27:04.447193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:27:04.447193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:27:04.447193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:27:04.447193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:27:04.447193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:27:04.447193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:27:04.447193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:27:04.447193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:27:04.447193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:27:04.447193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:27:04.447193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:27:04.447193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:27:04.448669 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:27:04.448669 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:27:04.448669 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:27:04.448669 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:27:04.448669 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:27:04.448669 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:27:04.448669 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:27:04.448669 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:27:04.448669 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 542/1104] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:27:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:27:04.453568 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca8f2b-16e8-4b97-b6a1-096cba4b2bb4 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:04.554012 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca8f2b-16e8-4b97-b6a1-096cba4b2bb4 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:04.587599 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f017855-d7f5-41ef-9b6d-d79a2eea317d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:04.588261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f017855-d7f5-41ef-9b6d-d79a2eea317d None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:04.588546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f017855-d7f5-41ef-9b6d-d79a2eea317d None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:04.588826 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f017855-d7f5-41ef-9b6d-d79a2eea317d None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:04.589610 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f017855-d7f5-41ef-9b6d-d79a2eea317d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:04.589925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f017855-d7f5-41ef-9b6d-d79a2eea317d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:04.591384 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f017855-d7f5-41ef-9b6d-d79a2eea317d None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:04.603594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 563/1105] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:27:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:04.609689 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecab9ec-5f81-48d7-b750-38e0f1f52c2a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:04.645499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:04.709761 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecab9ec-5f81-48d7-b750-38e0f1f52c2a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:04.742554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39915627-271b-4e59-a826-fbd1ab1292ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:04.743228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39915627-271b-4e59-a826-fbd1ab1292ac None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:04.743514 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39915627-271b-4e59-a826-fbd1ab1292ac None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:04.743791 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39915627-271b-4e59-a826-fbd1ab1292ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:04.745136 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39915627-271b-4e59-a826-fbd1ab1292ac None admin] RBAC: Adding query filter params (role.id=a9a87bede27348b89fa09b9dfd04e89c, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=161c5ed1e47e450290a1f428ee100b68) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:04.745449 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39915627-271b-4e59-a826-fbd1ab1292ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:04.746906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39915627-271b-4e59-a826-fbd1ab1292ac None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:04.752803 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 543/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:27:04 2022] GET /identity/v3/role_assignments?user.id=161c5ed1e47e450290a1f428ee100b68&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=a9a87bede27348b89fa09b9dfd04e89c => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:05.253834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79db80-a221-45a3-9f5a-969f3875a426 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:05.254346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79db80-a221-45a3-9f5a-969f3875a426 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:05.254462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79db80-a221-45a3-9f5a-969f3875a426 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:05.255795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 564/1107] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:27:05.261201 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339f2ec-95a9-4149-adf0-44a6477fb210 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:05.261663 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339f2ec-95a9-4149-adf0-44a6477fb210 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:05.261765 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339f2ec-95a9-4149-adf0-44a6477fb210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:05.278759 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f339f2ec-95a9-4149-adf0-44a6477fb210 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:27:05.279945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f339f2ec-95a9-4149-adf0-44a6477fb210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:05.348797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 544/1108] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:27:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:27:05.374135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79db80-a221-45a3-9f5a-969f3875a426 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:05.387068 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:05.477681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79db80-a221-45a3-9f5a-969f3875a426 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:05.513483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360f86d-444f-4c0c-875a-40f5a6045255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:05.514199 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360f86d-444f-4c0c-875a-40f5a6045255 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:05.514483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360f86d-444f-4c0c-875a-40f5a6045255 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:05.514792 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360f86d-444f-4c0c-875a-40f5a6045255 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:05.515945 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360f86d-444f-4c0c-875a-40f5a6045255 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:05.517365 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360f86d-444f-4c0c-875a-40f5a6045255 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:05.519010 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 565/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:27:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:05.524978 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339f2ec-95a9-4149-adf0-44a6477fb210 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:05.556415 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:05.628434 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339f2ec-95a9-4149-adf0-44a6477fb210 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:05.662168 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727948f-8bd2-4771-aa4d-5541149a1745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:05.662861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727948f-8bd2-4771-aa4d-5541149a1745 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:05.663145 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727948f-8bd2-4771-aa4d-5541149a1745 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:05.663421 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727948f-8bd2-4771-aa4d-5541149a1745 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:05.664128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f727948f-8bd2-4771-aa4d-5541149a1745 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:05.665563 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f727948f-8bd2-4771-aa4d-5541149a1745 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:05.748026 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 545/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 23 11:27:05 2022] POST /identity/v3/users => generated 286 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:27:07.823610 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c570c-b49a-434d-8819-5d52864a5eab None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:07.824130 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c570c-b49a-434d-8819-5d52864a5eab None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:07.824224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c570c-b49a-434d-8819-5d52864a5eab None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:07.825513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 566/1111] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:27:07.830610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f02eeb-3cc0-4f76-9a76-3655b0d29ac3 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:07.830998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f02eeb-3cc0-4f76-9a76-3655b0d29ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:07.831165 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f02eeb-3cc0-4f76-9a76-3655b0d29ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:07.848032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-81f02eeb-3cc0-4f76-9a76-3655b0d29ac3 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:27:07.849190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-81f02eeb-3cc0-4f76-9a76-3655b0d29ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:07.913877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 546/1112] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:27:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:27:07.938049 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c570c-b49a-434d-8819-5d52864a5eab None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:08.023399 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:08.045126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c570c-b49a-434d-8819-5d52864a5eab None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:08.078512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e9ee1-e28d-4863-add3-e88b6aa07ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:08.079427 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e9ee1-e28d-4863-add3-e88b6aa07ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:08.079715 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e9ee1-e28d-4863-add3-e88b6aa07ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:08.079985 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e9ee1-e28d-4863-add3-e88b6aa07ab9 None admin] PATH_INFO: `/v3/roles/25e99b2a08914aa0ab18cc4955f25364` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:08.081424 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e9ee1-e28d-4863-add3-e88b6aa07ab9 None admin] RBAC: Authorizing `identity:get_role(role_id=25e99b2a08914aa0ab18cc4955f25364)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:08.082802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e9ee1-e28d-4863-add3-e88b6aa07ab9 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:08.084147 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 567/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:27:07 2022] GET /identity/v3/roles/25e99b2a08914aa0ab18cc4955f25364 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:08.090620 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f02eeb-3cc0-4f76-9a76-3655b0d29ac3 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:08.107512 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:08.185534 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f02eeb-3cc0-4f76-9a76-3655b0d29ac3 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:08.218816 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987754ca-0870-492b-84f8-90f18175a9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:08.219673 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987754ca-0870-492b-84f8-90f18175a9a8 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:08.219955 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987754ca-0870-492b-84f8-90f18175a9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:08.220228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987754ca-0870-492b-84f8-90f18175a9a8 None admin] PATH_INFO: `/v3/users/9985fabf856a4c84aa6d6ac327737663` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:08.232792 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987754ca-0870-492b-84f8-90f18175a9a8 None admin] RBAC: Authorizing `identity:get_user(user_id=9985fabf856a4c84aa6d6ac327737663)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:08.234219 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987754ca-0870-492b-84f8-90f18175a9a8 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:08.235897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 547/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:27:08 2022] GET /identity/v3/users/9985fabf856a4c84aa6d6ac327737663 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:08.241470 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e9ee1-e28d-4863-add3-e88b6aa07ab9 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:08.338724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e9ee1-e28d-4863-add3-e88b6aa07ab9 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:08.371041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f6b0a-2e61-492e-b4f4-d6c0642c2e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:08.371669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f6b0a-2e61-492e-b4f4-d6c0642c2e7f None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:08.371949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f6b0a-2e61-492e-b4f4-d6c0642c2e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:08.372221 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f6b0a-2e61-492e-b4f4-d6c0642c2e7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:08.386463 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:08.437039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f6b0a-2e61-492e-b4f4-d6c0642c2e7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:08.438469 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f6b0a-2e61-492e-b4f4-d6c0642c2e7f None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:08.533208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 568/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:27:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:27:08.539007 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987754ca-0870-492b-84f8-90f18175a9a8 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:08.614377 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:08.638218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987754ca-0870-492b-84f8-90f18175a9a8 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:08.672833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0fa64f-72c5-4bee-bc54-52371f9802c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:08.673667 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fa64f-72c5-4bee-bc54-52371f9802c7 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:08.673945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fa64f-72c5-4bee-bc54-52371f9802c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:08.674217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fa64f-72c5-4bee-bc54-52371f9802c7 None admin] PATH_INFO: `/v3/projects/f2755c3c84a749d985413b51ff43d333` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:08.675399 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0fa64f-72c5-4bee-bc54-52371f9802c7 None admin] RBAC: Authorizing `identity:get_project(project_id=f2755c3c84a749d985413b51ff43d333)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:08.676812 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0fa64f-72c5-4bee-bc54-52371f9802c7 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:08.678468 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 548/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:27:08 2022] GET /identity/v3/projects/f2755c3c84a749d985413b51ff43d333 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:08.684494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f6b0a-2e61-492e-b4f4-d6c0642c2e7f None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:08.730059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:08.782111 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f6b0a-2e61-492e-b4f4-d6c0642c2e7f None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:08.814522 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b67ebc-2a41-449a-8e14-ef44057fd3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:08.815191 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b67ebc-2a41-449a-8e14-ef44057fd3db None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:08.815477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b67ebc-2a41-449a-8e14-ef44057fd3db None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:08.815756 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b67ebc-2a41-449a-8e14-ef44057fd3db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:08.816571 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b67ebc-2a41-449a-8e14-ef44057fd3db None admin] RBAC: Adding query filter params (role.id=25e99b2a08914aa0ab18cc4955f25364, scope.project.id=f2755c3c84a749d985413b51ff43d333, user.id=9985fabf856a4c84aa6d6ac327737663) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:08.817143 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b67ebc-2a41-449a-8e14-ef44057fd3db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:08.818613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b67ebc-2a41-449a-8e14-ef44057fd3db None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:08.825057 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 569/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:27:08 2022] GET /identity/v3/role_assignments?user.id=9985fabf856a4c84aa6d6ac327737663&scope.project.id=f2755c3c84a749d985413b51ff43d333&role.id=25e99b2a08914aa0ab18cc4955f25364 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:10.809449 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b71b0d-566a-4239-bb38-d0b84332ed9d None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:10.809950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b71b0d-566a-4239-bb38-d0b84332ed9d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:10.809950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b71b0d-566a-4239-bb38-d0b84332ed9d None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:10.811389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 549/1118] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:27:10.816579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df4af7-b011-4f16-b846-74c360b08a62 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:10.817210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df4af7-b011-4f16-b846-74c360b08a62 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:10.817392 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df4af7-b011-4f16-b846-74c360b08a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:10.834265 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-43df4af7-b011-4f16-b846-74c360b08a62 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:27:10.835503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-43df4af7-b011-4f16-b846-74c360b08a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:10.901231 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 570/1119] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:27:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:27:10.924426 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b71b0d-566a-4239-bb38-d0b84332ed9d None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:11.026221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b71b0d-566a-4239-bb38-d0b84332ed9d None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:11.058582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94202c-e520-4f76-8feb-7510a8cdb9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:11.059423 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94202c-e520-4f76-8feb-7510a8cdb9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:11.059757 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94202c-e520-4f76-8feb-7510a8cdb9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:11.059982 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94202c-e520-4f76-8feb-7510a8cdb9d1 None admin] PATH_INFO: `/v3/roles/25e99b2a08914aa0ab18cc4955f25364` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:11.061446 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94202c-e520-4f76-8feb-7510a8cdb9d1 None admin] RBAC: Authorizing `identity:get_role(role_id=25e99b2a08914aa0ab18cc4955f25364)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:11.062834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94202c-e520-4f76-8feb-7510a8cdb9d1 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:11.064191 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 550/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:27:10 2022] GET /identity/v3/roles/25e99b2a08914aa0ab18cc4955f25364 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:11.070268 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df4af7-b011-4f16-b846-74c360b08a62 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:11.110866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:11.167210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df4af7-b011-4f16-b846-74c360b08a62 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:11.200755 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6886dc-7ca4-4a76-827e-d3d542ee67e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:11.201600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6886dc-7ca4-4a76-827e-d3d542ee67e6 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:11.201878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6886dc-7ca4-4a76-827e-d3d542ee67e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:11.202148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6886dc-7ca4-4a76-827e-d3d542ee67e6 None admin] PATH_INFO: `/v3/users/9985fabf856a4c84aa6d6ac327737663` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:11.203324 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6886dc-7ca4-4a76-827e-d3d542ee67e6 None admin] RBAC: Authorizing `identity:get_user(user_id=9985fabf856a4c84aa6d6ac327737663)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:11.204739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6886dc-7ca4-4a76-827e-d3d542ee67e6 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:11.206330 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 571/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:27:11 2022] GET /identity/v3/users/9985fabf856a4c84aa6d6ac327737663 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:11.212082 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94202c-e520-4f76-8feb-7510a8cdb9d1 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:11.224582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:11.310464 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94202c-e520-4f76-8feb-7510a8cdb9d1 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:11.344583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85077239-15f0-4269-9cf4-417bf3b27138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:11.345427 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85077239-15f0-4269-9cf4-417bf3b27138 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:11.345712 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85077239-15f0-4269-9cf4-417bf3b27138 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:11.345990 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85077239-15f0-4269-9cf4-417bf3b27138 None admin] PATH_INFO: `/v3/projects/f2755c3c84a749d985413b51ff43d333` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:11.347242 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85077239-15f0-4269-9cf4-417bf3b27138 None admin] RBAC: Authorizing `identity:get_project(project_id=f2755c3c84a749d985413b51ff43d333)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:11.348682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85077239-15f0-4269-9cf4-417bf3b27138 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:11.350362 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 551/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:27:11 2022] GET /identity/v3/projects/f2755c3c84a749d985413b51ff43d333 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:11.356252 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6886dc-7ca4-4a76-827e-d3d542ee67e6 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:11.452185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6886dc-7ca4-4a76-827e-d3d542ee67e6 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:11.460869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:11.484019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0da1c-cb1b-415e-95fa-0c44e95fbd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:11.484896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0da1c-cb1b-415e-95fa-0c44e95fbd96 None admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:11.485175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0da1c-cb1b-415e-95fa-0c44e95fbd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:11.485443 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0da1c-cb1b-415e-95fa-0c44e95fbd96 None admin] PATH_INFO: `/v3/projects/f2755c3c84a749d985413b51ff43d333/users/9985fabf856a4c84aa6d6ac327737663/roles/25e99b2a08914aa0ab18cc4955f25364` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:11.487681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b0da1c-cb1b-415e-95fa-0c44e95fbd96 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2755c3c84a749d985413b51ff43d333, user_id=9985fabf856a4c84aa6d6ac327737663, role_id=25e99b2a08914aa0ab18cc4955f25364)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:11.489098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b0da1c-cb1b-415e-95fa-0c44e95fbd96 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:11.573974 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 572/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:27:11 2022] PUT /identity/v3/projects/f2755c3c84a749d985413b51ff43d333/users/9985fabf856a4c84aa6d6ac327737663/roles/25e99b2a08914aa0ab18cc4955f25364 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:27:13.577964 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e0eb5-1b44-4415-95a7-654ffec1eeac None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:13.578489 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e0eb5-1b44-4415-95a7-654ffec1eeac None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:13.578600 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e0eb5-1b44-4415-95a7-654ffec1eeac None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:13.579936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 552/1124] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:27:13.584985 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92927ac-dcf2-4263-9e71-159f1d2824bb None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:13.585385 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92927ac-dcf2-4263-9e71-159f1d2824bb None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:13.585533 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92927ac-dcf2-4263-9e71-159f1d2824bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:13.602144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-b92927ac-dcf2-4263-9e71-159f1d2824bb None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:27:13.604207 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b92927ac-dcf2-4263-9e71-159f1d2824bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:13.668171 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 573/1125] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:27:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:27:13.691526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e0eb5-1b44-4415-95a7-654ffec1eeac None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:13.769645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:13.792316 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e0eb5-1b44-4415-95a7-654ffec1eeac None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:13.825584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024ae66-fbc4-4424-9539-ac6d6497a07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:13.826426 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024ae66-fbc4-4424-9539-ac6d6497a07c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:13.826739 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024ae66-fbc4-4424-9539-ac6d6497a07c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:13.827013 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024ae66-fbc4-4424-9539-ac6d6497a07c None admin] PATH_INFO: `/v3/roles/25e99b2a08914aa0ab18cc4955f25364` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:13.828509 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024ae66-fbc4-4424-9539-ac6d6497a07c None admin] RBAC: Authorizing `identity:get_role(role_id=25e99b2a08914aa0ab18cc4955f25364)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:13.829882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024ae66-fbc4-4424-9539-ac6d6497a07c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:13.831264 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 553/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:27:13 2022] GET /identity/v3/roles/25e99b2a08914aa0ab18cc4955f25364 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:13.836911 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92927ac-dcf2-4263-9e71-159f1d2824bb None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:13.929124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:13.932767 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92927ac-dcf2-4263-9e71-159f1d2824bb None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:13.966331 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65e39a-e5ed-4f0e-b248-09daa12e1254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:13.967209 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65e39a-e5ed-4f0e-b248-09daa12e1254 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:13.967494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65e39a-e5ed-4f0e-b248-09daa12e1254 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:13.967770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65e39a-e5ed-4f0e-b248-09daa12e1254 None admin] PATH_INFO: `/v3/users/9985fabf856a4c84aa6d6ac327737663` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:13.968937 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e65e39a-e5ed-4f0e-b248-09daa12e1254 None admin] RBAC: Authorizing `identity:get_user(user_id=9985fabf856a4c84aa6d6ac327737663)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:13.970391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e65e39a-e5ed-4f0e-b248-09daa12e1254 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:13.972017 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 574/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:27:13 2022] GET /identity/v3/users/9985fabf856a4c84aa6d6ac327737663 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:13.977761 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024ae66-fbc4-4424-9539-ac6d6497a07c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:14.074947 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024ae66-fbc4-4424-9539-ac6d6497a07c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:14.107668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1357e-3e22-4234-9625-890b6bbeb42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:14.108299 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1357e-3e22-4234-9625-890b6bbeb42a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:14.108583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1357e-3e22-4234-9625-890b6bbeb42a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:14.108859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1357e-3e22-4234-9625-890b6bbeb42a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:14.123347 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:14.175266 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1357e-3e22-4234-9625-890b6bbeb42a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:14.176714 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1357e-3e22-4234-9625-890b6bbeb42a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:14.270446 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 554/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:27:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:27:14.277292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65e39a-e5ed-4f0e-b248-09daa12e1254 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:14.380969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65e39a-e5ed-4f0e-b248-09daa12e1254 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:14.413981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6942c-4705-4e8c-a9ab-3fa57a6aebdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:14.414847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6942c-4705-4e8c-a9ab-3fa57a6aebdc None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:14.415133 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6942c-4705-4e8c-a9ab-3fa57a6aebdc None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:14.415410 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6942c-4705-4e8c-a9ab-3fa57a6aebdc None admin] PATH_INFO: `/v3/projects/f2755c3c84a749d985413b51ff43d333` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:14.416588 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6942c-4705-4e8c-a9ab-3fa57a6aebdc None admin] RBAC: Authorizing `identity:get_project(project_id=f2755c3c84a749d985413b51ff43d333)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:14.418016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6942c-4705-4e8c-a9ab-3fa57a6aebdc None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:14.419723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 575/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:27:14 2022] GET /identity/v3/projects/f2755c3c84a749d985413b51ff43d333 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:14.425779 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1357e-3e22-4234-9625-890b6bbeb42a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:14.474918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:14.527223 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1357e-3e22-4234-9625-890b6bbeb42a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:14.559400 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a1917-cd36-44df-97b8-3a2740f45a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:14.560038 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a1917-cd36-44df-97b8-3a2740f45a19 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:14.560318 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a1917-cd36-44df-97b8-3a2740f45a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:14.560588 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a1917-cd36-44df-97b8-3a2740f45a19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:14.561405 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a1917-cd36-44df-97b8-3a2740f45a19 None admin] RBAC: Adding query filter params (role.id=25e99b2a08914aa0ab18cc4955f25364, scope.project.id=f2755c3c84a749d985413b51ff43d333, user.id=9985fabf856a4c84aa6d6ac327737663) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:14.561714 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a1917-cd36-44df-97b8-3a2740f45a19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:14.563157 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a1917-cd36-44df-97b8-3a2740f45a19 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:14.569061 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 555/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:27:14 2022] GET /identity/v3/role_assignments?user.id=9985fabf856a4c84aa6d6ac327737663&scope.project.id=f2755c3c84a749d985413b51ff43d333&role.id=25e99b2a08914aa0ab18cc4955f25364 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:16.602864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577c18b-fc9d-4b4d-ac5d-fa1c9613fd3b None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:16.603938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577c18b-fc9d-4b4d-ac5d-fa1c9613fd3b None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:16.604026 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577c18b-fc9d-4b4d-ac5d-fa1c9613fd3b None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:16.605372 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 576/1131] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:27:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:27:16.610456 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b78b1-8537-4fb0-9594-024f559040e4 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:16.610897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b78b1-8537-4fb0-9594-024f559040e4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:16.611040 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b78b1-8537-4fb0-9594-024f559040e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:16.627583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f47b78b1-8537-4fb0-9594-024f559040e4 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:27:16.628766 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f47b78b1-8537-4fb0-9594-024f559040e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:16.694248 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 556/1132] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:27:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:27:16.718083 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577c18b-fc9d-4b4d-ac5d-fa1c9613fd3b None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:16.751809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:16.816118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577c18b-fc9d-4b4d-ac5d-fa1c9613fd3b None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:16.848818 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004246d5-7d85-41c5-b4b5-6c377f46d6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:16.849593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004246d5-7d85-41c5-b4b5-6c377f46d6f0 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:16.849874 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004246d5-7d85-41c5-b4b5-6c377f46d6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:16.850147 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004246d5-7d85-41c5-b4b5-6c377f46d6f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:16.851390 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004246d5-7d85-41c5-b4b5-6c377f46d6f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:16.852819 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004246d5-7d85-41c5-b4b5-6c377f46d6f0 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:16.854433 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 577/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:27:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:16.860307 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b78b1-8537-4fb0-9594-024f559040e4 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:16.901304 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:16.957762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b78b1-8537-4fb0-9594-024f559040e4 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:16.990994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81904f24-c298-4bed-9f3f-a6e28e717d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:16.991637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81904f24-c298-4bed-9f3f-a6e28e717d4a None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:16.991917 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81904f24-c298-4bed-9f3f-a6e28e717d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:16.992190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81904f24-c298-4bed-9f3f-a6e28e717d4a None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:16.992885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81904f24-c298-4bed-9f3f-a6e28e717d4a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:16.994999 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81904f24-c298-4bed-9f3f-a6e28e717d4a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:17.071704 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 557/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 23 11:27:16 2022] POST /identity/v3/projects => generated 311 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:27:19.116435 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfbec7-64d2-47b4-99f2-c0732b616155 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:19.116969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfbec7-64d2-47b4-99f2-c0732b616155 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:19.116969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfbec7-64d2-47b4-99f2-c0732b616155 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:19.118335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 578/1135] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:27:19.123631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31453df-461a-4bf7-af29-71c4ece7f5e3 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:19.124053 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31453df-461a-4bf7-af29-71c4ece7f5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:19.124188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31453df-461a-4bf7-af29-71c4ece7f5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:19.140372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b31453df-461a-4bf7-af29-71c4ece7f5e3 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:27:19.141531 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b31453df-461a-4bf7-af29-71c4ece7f5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:19.205837 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 558/1136] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:27:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:27:19.230876 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfbec7-64d2-47b4-99f2-c0732b616155 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:19.328600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfbec7-64d2-47b4-99f2-c0732b616155 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:19.330094 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:19.361556 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87eea7-4f8d-44a6-8350-0c0001d86284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:19.362336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87eea7-4f8d-44a6-8350-0c0001d86284 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:19.362644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87eea7-4f8d-44a6-8350-0c0001d86284 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:19.362919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87eea7-4f8d-44a6-8350-0c0001d86284 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:19.364075 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a87eea7-4f8d-44a6-8350-0c0001d86284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:19.365502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a87eea7-4f8d-44a6-8350-0c0001d86284 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:19.367253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 579/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:27:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:19.373212 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31453df-461a-4bf7-af29-71c4ece7f5e3 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:19.396888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:19.471389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31453df-461a-4bf7-af29-71c4ece7f5e3 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:19.509130 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b43775-d945-472d-9b8c-37caa433bc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:19.510773 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b43775-d945-472d-9b8c-37caa433bc1c None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:19.511056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b43775-d945-472d-9b8c-37caa433bc1c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:19.511324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b43775-d945-472d-9b8c-37caa433bc1c None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:19.512027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b43775-d945-472d-9b8c-37caa433bc1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:19.513445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b43775-d945-472d-9b8c-37caa433bc1c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:19.593298 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 559/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 23 11:27:19 2022] POST /identity/v3/users => generated 286 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:27:21.680887 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c175c-2c41-42be-88a4-07c999fa0f6c None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:21.681444 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c175c-2c41-42be-88a4-07c999fa0f6c None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:21.681642 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c175c-2c41-42be-88a4-07c999fa0f6c None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:21.683057 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 580/1139] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:27:21.688139 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8a0e6-56cf-4e74-832d-06312cb3b6d7 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:21.688513 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8a0e6-56cf-4e74-832d-06312cb3b6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:21.688693 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8a0e6-56cf-4e74-832d-06312cb3b6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:21.706137 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-aca8a0e6-56cf-4e74-832d-06312cb3b6d7 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:27:21.707347 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-aca8a0e6-56cf-4e74-832d-06312cb3b6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:21.771853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 560/1140] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:27:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:27:21.795510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c175c-2c41-42be-88a4-07c999fa0f6c None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:21.892124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c175c-2c41-42be-88a4-07c999fa0f6c None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:21.924307 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35fcc9-6b58-47d7-9d55-f7bde4b8745b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:21.925075 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35fcc9-6b58-47d7-9d55-f7bde4b8745b None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:21.925356 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35fcc9-6b58-47d7-9d55-f7bde4b8745b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:21.925627 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35fcc9-6b58-47d7-9d55-f7bde4b8745b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:21.935906 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35fcc9-6b58-47d7-9d55-f7bde4b8745b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:27:21.943579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-4d35fcc9-6b58-47d7-9d55-f7bde4b8745b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:27:21.943579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:27:21.943579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:27:21.943579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:27:21.943579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:27:21.943579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:27:21.943579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:27:21.943579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:27:21.943579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:27:21.943579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:27:21.943579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:27:21.943579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:27:21.943579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:27:21.943579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:27:21.943579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:27:21.943579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:27:21.944529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:27:21.944529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:27:21.944529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:27:21.944529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:27:21.944529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:27:21.944529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:27:21.944529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:27:21.944529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:27:21.944529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:27:21.944529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:27:21.944529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:27:21.944529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:27:21.944529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:27:21.944529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:27:21.944529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:27:21.944529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:27:21.945411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:27:21.945411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:27:21.945411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:27:21.945411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:27:21.945411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:27:21.945411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:27:21.945411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:27:21.945411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:27:21.945411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:27:21.945411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:27:21.945411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:27:21.945411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:27:21.945411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 581/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:27:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:27:21.951180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8a0e6-56cf-4e74-832d-06312cb3b6d7 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:21.961071 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:22.046858 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8a0e6-56cf-4e74-832d-06312cb3b6d7 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:22.081046 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b0119-b663-43f3-9532-612014145c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:22.081704 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b0119-b663-43f3-9532-612014145c4e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:22.081992 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b0119-b663-43f3-9532-612014145c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:22.082265 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b0119-b663-43f3-9532-612014145c4e None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:22.083043 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b0119-b663-43f3-9532-612014145c4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:22.083351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b0119-b663-43f3-9532-612014145c4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:22.084706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b0119-b663-43f3-9532-612014145c4e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:22.092507 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 561/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:27:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:22.098141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35fcc9-6b58-47d7-9d55-f7bde4b8745b None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:22.127773 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:22.195973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35fcc9-6b58-47d7-9d55-f7bde4b8745b None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:22.228724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8aafc-856f-4471-9eb6-fb0ec1b7b986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:22.229589 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8aafc-856f-4471-9eb6-fb0ec1b7b986 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:22.229875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8aafc-856f-4471-9eb6-fb0ec1b7b986 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:22.230152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8aafc-856f-4471-9eb6-fb0ec1b7b986 None admin] PATH_INFO: `/v3/users/8e42ab79495a405fba6d0871b1c5e130` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:22.242782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8aafc-856f-4471-9eb6-fb0ec1b7b986 None admin] RBAC: Authorizing `identity:get_user(user_id=8e42ab79495a405fba6d0871b1c5e130)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:22.244937 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8aafc-856f-4471-9eb6-fb0ec1b7b986 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:22.246456 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 582/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:27:22 2022] GET /identity/v3/users/8e42ab79495a405fba6d0871b1c5e130 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:22.252057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b0119-b663-43f3-9532-612014145c4e None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:22.320070 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:22.350543 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b0119-b663-43f3-9532-612014145c4e None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:22.383804 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45363b8f-bbd7-4b8c-ad6f-6aeccc2e2987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:22.384611 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45363b8f-bbd7-4b8c-ad6f-6aeccc2e2987 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:22.384943 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45363b8f-bbd7-4b8c-ad6f-6aeccc2e2987 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:22.385225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45363b8f-bbd7-4b8c-ad6f-6aeccc2e2987 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:22.456030 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45363b8f-bbd7-4b8c-ad6f-6aeccc2e2987 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:22.457485 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45363b8f-bbd7-4b8c-ad6f-6aeccc2e2987 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:22.523856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:22.554673 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 562/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:27:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:27:22.560944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8aafc-856f-4471-9eb6-fb0ec1b7b986 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:22.645600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:22.657510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8aafc-856f-4471-9eb6-fb0ec1b7b986 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:22.690734 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac37721-e148-4ace-a3b8-f696b5b6b3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:22.691817 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac37721-e148-4ace-a3b8-f696b5b6b3ad None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:22.692098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac37721-e148-4ace-a3b8-f696b5b6b3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:22.692373 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac37721-e148-4ace-a3b8-f696b5b6b3ad None admin] PATH_INFO: `/v3/projects/7031f10d58024cad9f8e17db82c3d403` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:22.693534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac37721-e148-4ace-a3b8-f696b5b6b3ad None admin] RBAC: Authorizing `identity:get_project(project_id=7031f10d58024cad9f8e17db82c3d403)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:22.694998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac37721-e148-4ace-a3b8-f696b5b6b3ad None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:22.696757 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 583/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:27:22 2022] GET /identity/v3/projects/7031f10d58024cad9f8e17db82c3d403 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:22.702669 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45363b8f-bbd7-4b8c-ad6f-6aeccc2e2987 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:22.797479 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45363b8f-bbd7-4b8c-ad6f-6aeccc2e2987 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:22.829499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a4be4-8b06-4e1b-a834-4196498b0ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:22.830133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a4be4-8b06-4e1b-a834-4196498b0ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:22.830412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a4be4-8b06-4e1b-a834-4196498b0ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:22.830709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a4be4-8b06-4e1b-a834-4196498b0ff3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:22.831519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a4be4-8b06-4e1b-a834-4196498b0ff3 None admin] RBAC: Adding query filter params (role.id=c9e006de0d6c4812ab29d778e9c8a830, scope.project.id=7031f10d58024cad9f8e17db82c3d403, user.id=8e42ab79495a405fba6d0871b1c5e130) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:22.831830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a4be4-8b06-4e1b-a834-4196498b0ff3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:22.833236 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a4be4-8b06-4e1b-a834-4196498b0ff3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:22.839108 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 563/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:27:22 2022] GET /identity/v3/role_assignments?user.id=8e42ab79495a405fba6d0871b1c5e130&scope.project.id=7031f10d58024cad9f8e17db82c3d403&role.id=c9e006de0d6c4812ab29d778e9c8a830 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:24.814782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce01e52-46b3-40bc-b830-3dcbdbdf626b None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:24.815291 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce01e52-46b3-40bc-b830-3dcbdbdf626b None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:24.815291 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce01e52-46b3-40bc-b830-3dcbdbdf626b None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:24.816874 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 584/1147] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:27:24.822176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39351e55-58f9-47fe-91dd-793b234c4279 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:24.822619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39351e55-58f9-47fe-91dd-793b234c4279 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:24.822767 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39351e55-58f9-47fe-91dd-793b234c4279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:24.839731 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-39351e55-58f9-47fe-91dd-793b234c4279 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:27:24.840912 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-39351e55-58f9-47fe-91dd-793b234c4279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:24.867707 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:24.909824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 564/1148] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:27:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:27:24.933753 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce01e52-46b3-40bc-b830-3dcbdbdf626b None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:25.037214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce01e52-46b3-40bc-b830-3dcbdbdf626b None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:25.070761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa34a61-768e-4c1c-a358-9b72a937dc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:25.071595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa34a61-768e-4c1c-a358-9b72a937dc2f None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:25.072331 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa34a61-768e-4c1c-a358-9b72a937dc2f None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:25.072610 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa34a61-768e-4c1c-a358-9b72a937dc2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:25.082221 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa34a61-768e-4c1c-a358-9b72a937dc2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:27:25.090632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-0fa34a61-768e-4c1c-a358-9b72a937dc2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:27:25.090632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:27:25.090632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:27:25.090632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:27:25.090632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:27:25.090632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:27:25.090632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:27:25.090632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:27:25.090632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:27:25.090632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:27:25.090632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:27:25.090632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:27:25.090632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:27:25.090632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:27:25.090632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:27:25.090632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:27:25.091825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:27:25.091825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:27:25.091825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:27:25.091825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:27:25.091825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:27:25.091825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:27:25.091825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:27:25.091825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:27:25.091825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:27:25.091825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:27:25.091825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:27:25.091825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:27:25.091825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:27:25.091825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:27:25.091825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:27:25.091825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:27:25.092886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:27:25.092886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:27:25.092886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:27:25.092886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:27:25.092886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:27:25.092886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:27:25.092886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:27:25.092886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:27:25.092886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:27:25.092886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:27:25.092886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:27:25.092886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:27:25.092886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 585/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:27:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:27:25.097916 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39351e55-58f9-47fe-91dd-793b234c4279 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:25.195212 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39351e55-58f9-47fe-91dd-793b234c4279 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:25.229031 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57e513-ab6c-460a-b763-c4ad61bc6477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:25.229682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57e513-ab6c-460a-b763-c4ad61bc6477 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:25.229961 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57e513-ab6c-460a-b763-c4ad61bc6477 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:25.230230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57e513-ab6c-460a-b763-c4ad61bc6477 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:25.230989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57e513-ab6c-460a-b763-c4ad61bc6477 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:25.231293 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57e513-ab6c-460a-b763-c4ad61bc6477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:25.232629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57e513-ab6c-460a-b763-c4ad61bc6477 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:25.240274 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 565/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:27:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:25.245853 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa34a61-768e-4c1c-a358-9b72a937dc2f None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:25.264318 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:25.344965 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa34a61-768e-4c1c-a358-9b72a937dc2f None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:25.377205 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77d34d-5f1b-4383-9b90-e2af6ca0765f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:25.378114 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77d34d-5f1b-4383-9b90-e2af6ca0765f None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:25.378395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77d34d-5f1b-4383-9b90-e2af6ca0765f None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:25.378694 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77d34d-5f1b-4383-9b90-e2af6ca0765f None admin] PATH_INFO: `/v3/users/8e42ab79495a405fba6d0871b1c5e130` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:25.379856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77d34d-5f1b-4383-9b90-e2af6ca0765f None admin] RBAC: Authorizing `identity:get_user(user_id=8e42ab79495a405fba6d0871b1c5e130)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:25.381269 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77d34d-5f1b-4383-9b90-e2af6ca0765f None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:25.382889 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 586/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:27:25 2022] GET /identity/v3/users/8e42ab79495a405fba6d0871b1c5e130 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:25.389098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57e513-ab6c-460a-b763-c4ad61bc6477 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:25.416454 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:25.493214 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57e513-ab6c-460a-b763-c4ad61bc6477 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:25.529036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b23af-6258-4f5e-a103-b0c472b7ad65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:25.529881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b23af-6258-4f5e-a103-b0c472b7ad65 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:25.530161 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b23af-6258-4f5e-a103-b0c472b7ad65 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:25.530430 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b23af-6258-4f5e-a103-b0c472b7ad65 None admin] PATH_INFO: `/v3/projects/7031f10d58024cad9f8e17db82c3d403` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:25.531617 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b23af-6258-4f5e-a103-b0c472b7ad65 None admin] RBAC: Authorizing `identity:get_project(project_id=7031f10d58024cad9f8e17db82c3d403)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:25.533033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b23af-6258-4f5e-a103-b0c472b7ad65 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:25.534727 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 566/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:27:25 2022] GET /identity/v3/projects/7031f10d58024cad9f8e17db82c3d403 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:25.540724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77d34d-5f1b-4383-9b90-e2af6ca0765f None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:25.622512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:25.637894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77d34d-5f1b-4383-9b90-e2af6ca0765f None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:25.670222 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b3a4a-00e6-446e-b19a-e681255b63ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:25.671120 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b3a4a-00e6-446e-b19a-e681255b63ab None admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:25.671402 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b3a4a-00e6-446e-b19a-e681255b63ab None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:25.671674 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b3a4a-00e6-446e-b19a-e681255b63ab None admin] PATH_INFO: `/v3/projects/7031f10d58024cad9f8e17db82c3d403/users/8e42ab79495a405fba6d0871b1c5e130/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:25.673596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b3a4a-00e6-446e-b19a-e681255b63ab None admin] RBAC: Authorizing `identity:create_grant(project_id=7031f10d58024cad9f8e17db82c3d403, user_id=8e42ab79495a405fba6d0871b1c5e130, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:25.675053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b3a4a-00e6-446e-b19a-e681255b63ab None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:25.766771 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 587/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:27:25 2022] PUT /identity/v3/projects/7031f10d58024cad9f8e17db82c3d403/users/8e42ab79495a405fba6d0871b1c5e130/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:27:27.789590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9599b3-06bb-44c8-8cc2-508adafd91c6 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:27.790708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9599b3-06bb-44c8-8cc2-508adafd91c6 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:27.790708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9599b3-06bb-44c8-8cc2-508adafd91c6 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:27.791975 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 567/1154] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:27:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:27:27.797202 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0f368-aecc-4540-aa9f-447f0452e400 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:27.797623 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0f368-aecc-4540-aa9f-447f0452e400 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:27.797751 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0f368-aecc-4540-aa9f-447f0452e400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:27.815178 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-bcf0f368-aecc-4540-aa9f-447f0452e400 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:27:27.816338 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-bcf0f368-aecc-4540-aa9f-447f0452e400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:27.881287 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 588/1155] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:27:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:27:27.905515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9599b3-06bb-44c8-8cc2-508adafd91c6 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:27.998080 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:28.005208 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9599b3-06bb-44c8-8cc2-508adafd91c6 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:28.039251 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689c6cd-aed9-4463-a427-a0f111b2747b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:28.040074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689c6cd-aed9-4463-a427-a0f111b2747b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:28.041215 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689c6cd-aed9-4463-a427-a0f111b2747b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:28.041492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689c6cd-aed9-4463-a427-a0f111b2747b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:28.051170 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689c6cd-aed9-4463-a427-a0f111b2747b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:27:28.059140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-2689c6cd-aed9-4463-a427-a0f111b2747b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:27:28.059140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:27:28.059140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:27:28.059140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:27:28.059140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:27:28.059140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:27:28.059140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:27:28.059140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:27:28.059140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:27:28.059140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:27:28.059140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:27:28.059140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:27:28.059140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:27:28.059140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:27:28.059140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:27:28.059140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:27:28.060287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:27:28.060287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:27:28.060287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:27:28.060287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:27:28.060287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:27:28.060287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:27:28.060287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:27:28.060287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:27:28.060287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:27:28.060287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:27:28.060287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:27:28.060287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:27:28.060287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:27:28.060287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:27:28.060287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:27:28.060287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:27:28.061499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:27:28.061499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:27:28.061499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:27:28.061499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:27:28.061499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:27:28.061499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:27:28.061499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:27:28.061499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:27:28.061499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:27:28.061499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:27:28.061499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:27:28.061499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:27:28.061499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 568/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:27:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:27:28.066132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0f368-aecc-4540-aa9f-447f0452e400 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:28.139924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:28.164163 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0f368-aecc-4540-aa9f-447f0452e400 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:28.198207 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7521117-644f-47e0-9cc5-098100cf7a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:28.198887 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7521117-644f-47e0-9cc5-098100cf7a4c None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:28.199168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7521117-644f-47e0-9cc5-098100cf7a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:28.199439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7521117-644f-47e0-9cc5-098100cf7a4c None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:28.200172 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7521117-644f-47e0-9cc5-098100cf7a4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:28.200476 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7521117-644f-47e0-9cc5-098100cf7a4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:28.201860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7521117-644f-47e0-9cc5-098100cf7a4c None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:28.209579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 589/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:27:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:28.215896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689c6cd-aed9-4463-a427-a0f111b2747b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:28.311440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689c6cd-aed9-4463-a427-a0f111b2747b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:28.344079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5a52b-df16-4fa6-a718-69a894638627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:28.344929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5a52b-df16-4fa6-a718-69a894638627 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:28.345204 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5a52b-df16-4fa6-a718-69a894638627 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:28.345473 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5a52b-df16-4fa6-a718-69a894638627 None admin] PATH_INFO: `/v3/users/8e42ab79495a405fba6d0871b1c5e130` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:28.346657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5a52b-df16-4fa6-a718-69a894638627 None admin] RBAC: Authorizing `identity:get_user(user_id=8e42ab79495a405fba6d0871b1c5e130)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:28.348111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5a52b-df16-4fa6-a718-69a894638627 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:28.349691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 569/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:27:28 2022] GET /identity/v3/users/8e42ab79495a405fba6d0871b1c5e130 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:28.355646 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7521117-644f-47e0-9cc5-098100cf7a4c None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:28.451327 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7521117-644f-47e0-9cc5-098100cf7a4c None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:28.483359 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f43185-6eec-4090-8814-66c945d93fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:28.483991 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f43185-6eec-4090-8814-66c945d93fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:28.484273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f43185-6eec-4090-8814-66c945d93fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:28.484545 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f43185-6eec-4090-8814-66c945d93fd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:28.490615 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:28.552460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f43185-6eec-4090-8814-66c945d93fd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:28.553944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f43185-6eec-4090-8814-66c945d93fd1 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:28.648308 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 590/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:27:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:27:28.654082 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5a52b-df16-4fa6-a718-69a894638627 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:28.678010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:28.751763 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5a52b-df16-4fa6-a718-69a894638627 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:28.784830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072e4fe-d469-4aea-95bf-3c1a94344d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:28.785667 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072e4fe-d469-4aea-95bf-3c1a94344d37 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:28.785955 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072e4fe-d469-4aea-95bf-3c1a94344d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:28.786227 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072e4fe-d469-4aea-95bf-3c1a94344d37 None admin] PATH_INFO: `/v3/projects/7031f10d58024cad9f8e17db82c3d403` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:28.787499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072e4fe-d469-4aea-95bf-3c1a94344d37 None admin] RBAC: Authorizing `identity:get_project(project_id=7031f10d58024cad9f8e17db82c3d403)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:28.788926 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072e4fe-d469-4aea-95bf-3c1a94344d37 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:28.790675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 570/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:27:28 2022] GET /identity/v3/projects/7031f10d58024cad9f8e17db82c3d403 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:28.796598 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f43185-6eec-4090-8814-66c945d93fd1 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:28.831093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:28.892829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f43185-6eec-4090-8814-66c945d93fd1 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:28.925743 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160388a-4f2f-42ff-9085-f6528470d316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:28.926326 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160388a-4f2f-42ff-9085-f6528470d316 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:28.926632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160388a-4f2f-42ff-9085-f6528470d316 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:28.926909 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160388a-4f2f-42ff-9085-f6528470d316 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:28.927729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6160388a-4f2f-42ff-9085-f6528470d316 None admin] RBAC: Adding query filter params (role.id=c9e006de0d6c4812ab29d778e9c8a830, scope.project.id=7031f10d58024cad9f8e17db82c3d403, user.id=8e42ab79495a405fba6d0871b1c5e130) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:28.928039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6160388a-4f2f-42ff-9085-f6528470d316 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:28.929456 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6160388a-4f2f-42ff-9085-f6528470d316 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:28.935804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 591/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:27:28 2022] GET /identity/v3/role_assignments?user.id=8e42ab79495a405fba6d0871b1c5e130&scope.project.id=7031f10d58024cad9f8e17db82c3d403&role.id=c9e006de0d6c4812ab29d778e9c8a830 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:30.967357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea91dc5-34ba-4f18-a5ca-bb7bf92c2cb0 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:30.967910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea91dc5-34ba-4f18-a5ca-bb7bf92c2cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:30.967910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea91dc5-34ba-4f18-a5ca-bb7bf92c2cb0 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:30.969306 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 571/1162] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:27:30.974325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82658a-97de-4ffa-b143-90d089800028 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:30.974770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82658a-97de-4ffa-b143-90d089800028 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:30.974926 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82658a-97de-4ffa-b143-90d089800028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:30.991258 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-7a82658a-97de-4ffa-b143-90d089800028 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:27:30.992432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-7a82658a-97de-4ffa-b143-90d089800028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:31.056985 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 592/1163] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:27:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:27:31.079867 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea91dc5-34ba-4f18-a5ca-bb7bf92c2cb0 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:31.168921 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:31.178720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea91dc5-34ba-4f18-a5ca-bb7bf92c2cb0 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:31.213090 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9912a24-26fe-4608-a034-2baed2fe6238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:31.213720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9912a24-26fe-4608-a034-2baed2fe6238 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:31.213999 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9912a24-26fe-4608-a034-2baed2fe6238 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:31.214272 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9912a24-26fe-4608-a034-2baed2fe6238 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:31.279584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9912a24-26fe-4608-a034-2baed2fe6238 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:31.280952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9912a24-26fe-4608-a034-2baed2fe6238 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:31.366310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:31.376958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 572/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:27:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:27:31.383026 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82658a-97de-4ffa-b143-90d089800028 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:31.412658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:31.480904 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82658a-97de-4ffa-b143-90d089800028 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:31.514183 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d203039-d88f-49fa-a20e-0cd9e1a4a8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:31.515010 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d203039-d88f-49fa-a20e-0cd9e1a4a8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:31.515297 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d203039-d88f-49fa-a20e-0cd9e1a4a8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:31.515576 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d203039-d88f-49fa-a20e-0cd9e1a4a8b3 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:31.522052 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d203039-d88f-49fa-a20e-0cd9e1a4a8b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:31.523514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d203039-d88f-49fa-a20e-0cd9e1a4a8b3 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:31.537316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-6d203039-d88f-49fa-a20e-0cd9e1a4a8b3 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 11:27:31.537316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:27:31.537316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 23 11:27:31.537316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 23 11:27:31.537316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:27:31.537316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:27:31.537316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:27:31.537316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:27:31.537316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 23 11:27:31.537316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:27:31.537316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:27:31.537316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:27:31.537316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:27:31.537316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:27:31.537316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:27:31.537316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:27:31.537316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:27:31.538478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:27:31.538478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:27:31.538478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:27:31.538478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:27:31.538478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:27:31.538478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:27:31.538478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:27:31.538478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:27:31.538478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:27:31.538478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:27:31.538478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:27:31.538478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:27:31.538478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:27:31.538478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:27:31.538478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:27:31.538478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:27:31.539823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:27:31.539823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:27:31.539823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:27:31.539823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:27:31.539823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:27:31.539823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:27:31.539823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:27:31.539823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:27:31.539823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:27:31.539823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:27:31.539823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 23 11:27:31.539823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:27:31.539823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 11:27:31.539823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:27:31.539823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 593/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jun 23 11:27:31 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:27:31.545300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9912a24-26fe-4608-a034-2baed2fe6238 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:31.642577 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9912a24-26fe-4608-a034-2baed2fe6238 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:31.677027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d6ba5-8ec4-4451-80f8-5eb3c5ced405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:31.677666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d6ba5-8ec4-4451-80f8-5eb3c5ced405 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:31.677942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d6ba5-8ec4-4451-80f8-5eb3c5ced405 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:31.678216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d6ba5-8ec4-4451-80f8-5eb3c5ced405 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:31.678949 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d6ba5-8ec4-4451-80f8-5eb3c5ced405 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:31.679252 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d6ba5-8ec4-4451-80f8-5eb3c5ced405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:31.680983 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d6ba5-8ec4-4451-80f8-5eb3c5ced405 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:31.687819 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 573/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jun 23 11:27:31 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:33.732780 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2405a57-5c8b-4784-b82e-049d5745a204 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:33.733352 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2405a57-5c8b-4784-b82e-049d5745a204 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:33.733352 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2405a57-5c8b-4784-b82e-049d5745a204 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:33.734739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 594/1167] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:27:33.739963 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6e7d6-5133-4e79-9bad-c2c5c2886081 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:33.740396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6e7d6-5133-4e79-9bad-c2c5c2886081 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:33.740488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6e7d6-5133-4e79-9bad-c2c5c2886081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:33.756827 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-45e6e7d6-5133-4e79-9bad-c2c5c2886081 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:27:33.758009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-45e6e7d6-5133-4e79-9bad-c2c5c2886081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:33.808084 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:33.825951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 574/1168] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:27:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:27:33.848639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2405a57-5c8b-4784-b82e-049d5745a204 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:33.940420 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:33.951948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2405a57-5c8b-4784-b82e-049d5745a204 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:33.985741 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6523921-70aa-4918-8c14-f6dc02a8d159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:33.986395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6523921-70aa-4918-8c14-f6dc02a8d159 None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:33.986718 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6523921-70aa-4918-8c14-f6dc02a8d159 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:33.986998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6523921-70aa-4918-8c14-f6dc02a8d159 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:33.987749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6523921-70aa-4918-8c14-f6dc02a8d159 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:33.989297 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6523921-70aa-4918-8c14-f6dc02a8d159 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:34.059690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 595/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Thu Jun 23 11:27:33 2022] POST /identity/v3/domains => generated 267 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:27:36.134428 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c863f-69f5-420f-ae5d-7faa6111d0c2 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:36.135017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c863f-69f5-420f-ae5d-7faa6111d0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:36.135017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c863f-69f5-420f-ae5d-7faa6111d0c2 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:36.136373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 575/1170] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:27:36.141631 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490662f-aa8f-48c7-8171-ba8b0d4e50a8 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:36.142007 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490662f-aa8f-48c7-8171-ba8b0d4e50a8 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:36.142240 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490662f-aa8f-48c7-8171-ba8b0d4e50a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:36.159220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-5490662f-aa8f-48c7-8171-ba8b0d4e50a8 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:27:36.160386 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-5490662f-aa8f-48c7-8171-ba8b0d4e50a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:36.225180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 596/1171] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:27:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:27:36.248602 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c863f-69f5-420f-ae5d-7faa6111d0c2 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:36.346783 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c863f-69f5-420f-ae5d-7faa6111d0c2 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:36.379399 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9082bc5-92e6-416a-a530-61f78f1a2aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:36.380245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9082bc5-92e6-416a-a530-61f78f1a2aba None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:36.380524 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9082bc5-92e6-416a-a530-61f78f1a2aba None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:36.380795 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9082bc5-92e6-416a-a530-61f78f1a2aba None admin] PATH_INFO: `/v3/domains/dd476ddc74b942eb91ddc72f46689e53` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:36.390913 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9082bc5-92e6-416a-a530-61f78f1a2aba None admin] RBAC: Authorizing `identity:get_domain(domain_id=dd476ddc74b942eb91ddc72f46689e53)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:36.392286 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9082bc5-92e6-416a-a530-61f78f1a2aba None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:36.393884 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 576/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Thu Jun 23 11:27:36 2022] GET /identity/v3/domains/dd476ddc74b942eb91ddc72f46689e53 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:36.400562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490662f-aa8f-48c7-8171-ba8b0d4e50a8 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:36.488334 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:36.506347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490662f-aa8f-48c7-8171-ba8b0d4e50a8 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:36.539506 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c893b-1aef-45e1-b4c7-9ef6e65dce40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:36.540190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c893b-1aef-45e1-b4c7-9ef6e65dce40 None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:36.540465 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c893b-1aef-45e1-b4c7-9ef6e65dce40 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:36.540736 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c893b-1aef-45e1-b4c7-9ef6e65dce40 None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:36.541432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c893b-1aef-45e1-b4c7-9ef6e65dce40 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:36.543000 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c893b-1aef-45e1-b4c7-9ef6e65dce40 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:36.647867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 597/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 23 11:27:36 2022] POST /identity/v3/projects => generated 361 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:27:38.695475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62541475-bf24-47fc-abc6-5b8d2efb5b96 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:38.695999 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62541475-bf24-47fc-abc6-5b8d2efb5b96 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:38.696178 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62541475-bf24-47fc-abc6-5b8d2efb5b96 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:38.697556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 577/1174] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:27:38.702789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a6670-a7d8-44f7-b60d-3ad6a3aea7c5 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:38.703220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a6670-a7d8-44f7-b60d-3ad6a3aea7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:38.703360 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a6670-a7d8-44f7-b60d-3ad6a3aea7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:38.719731 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-fb5a6670-a7d8-44f7-b60d-3ad6a3aea7c5 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:27:38.720912 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-fb5a6670-a7d8-44f7-b60d-3ad6a3aea7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:38.790328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 598/1175] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:27:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:27:38.823294 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62541475-bf24-47fc-abc6-5b8d2efb5b96 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:38.869797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:38.923021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62541475-bf24-47fc-abc6-5b8d2efb5b96 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:38.956287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d8b06-1d24-472c-803f-761508056c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:38.957142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d8b06-1d24-472c-803f-761508056c24 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:38.957428 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d8b06-1d24-472c-803f-761508056c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:38.957700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d8b06-1d24-472c-803f-761508056c24 None admin] PATH_INFO: `/v3/domains/dd476ddc74b942eb91ddc72f46689e53` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:38.958905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d8b06-1d24-472c-803f-761508056c24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=dd476ddc74b942eb91ddc72f46689e53)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:38.960336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d8b06-1d24-472c-803f-761508056c24 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:38.961947 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 578/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Thu Jun 23 11:27:38 2022] GET /identity/v3/domains/dd476ddc74b942eb91ddc72f46689e53 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:38.967737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a6670-a7d8-44f7-b60d-3ad6a3aea7c5 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:39.026438 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:39.063868 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a6670-a7d8-44f7-b60d-3ad6a3aea7c5 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:39.097154 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dade910-1b7a-48aa-84af-74401d322894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:39.097805 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dade910-1b7a-48aa-84af-74401d322894 None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:39.098084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dade910-1b7a-48aa-84af-74401d322894 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:39.098355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dade910-1b7a-48aa-84af-74401d322894 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:39.099081 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dade910-1b7a-48aa-84af-74401d322894 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:39.100504 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dade910-1b7a-48aa-84af-74401d322894 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:39.172176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 599/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 23 11:27:38 2022] POST /identity/v3/users => generated 311 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:27:41.208173 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4fb40-4330-4f34-8a6a-58f081dca8cb None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:41.208728 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4fb40-4330-4f34-8a6a-58f081dca8cb None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:41.208728 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4fb40-4330-4f34-8a6a-58f081dca8cb None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:41.210073 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 579/1178] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:27:41.215707 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5383ba8-c48a-4ab1-9102-538d1a67c20c None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:41.216107 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5383ba8-c48a-4ab1-9102-538d1a67c20c None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:41.216255 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5383ba8-c48a-4ab1-9102-538d1a67c20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:41.233330 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-e5383ba8-c48a-4ab1-9102-538d1a67c20c None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:27:41.234484 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e5383ba8-c48a-4ab1-9102-538d1a67c20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:41.299558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 600/1179] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:27:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:27:41.324117 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4fb40-4330-4f34-8a6a-58f081dca8cb None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:41.422822 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4fb40-4330-4f34-8a6a-58f081dca8cb None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:41.435354 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:41.455084 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7797022-8ddb-4d09-a580-033f4adc4762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:41.455855 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7797022-8ddb-4d09-a580-033f4adc4762 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:41.456133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7797022-8ddb-4d09-a580-033f4adc4762 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:41.456401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7797022-8ddb-4d09-a580-033f4adc4762 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:41.466697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7797022-8ddb-4d09-a580-033f4adc4762 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:27:41.474057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-c7797022-8ddb-4d09-a580-033f4adc4762 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:27:41.474057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:27:41.474057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:27:41.474057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:27:41.474057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:27:41.474057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:27:41.474057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:27:41.474057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:27:41.474057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:27:41.474057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:27:41.474057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:27:41.474057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:27:41.474057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:27:41.474057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:27:41.474057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:27:41.474057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:27:41.475232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:27:41.475232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:27:41.475232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:27:41.475232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:27:41.475232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:27:41.475232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:27:41.475232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:27:41.475232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:27:41.475232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:27:41.475232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:27:41.475232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:27:41.475232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:27:41.475232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:27:41.475232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:27:41.475232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:27:41.475232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:27:41.476166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:27:41.476166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:27:41.476166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:27:41.476166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:27:41.476166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:27:41.476166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:27:41.476166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:27:41.476166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:27:41.476166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:27:41.476166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:27:41.476166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:27:41.476166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:27:41.476940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 580/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:27:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:27:41.481871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5383ba8-c48a-4ab1-9102-538d1a67c20c None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:41.528712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:41.577111 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5383ba8-c48a-4ab1-9102-538d1a67c20c None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:41.610951 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0265455-d1f4-499f-a391-42f45fd4ac30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:41.611603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0265455-d1f4-499f-a391-42f45fd4ac30 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:41.611885 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0265455-d1f4-499f-a391-42f45fd4ac30 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:41.612164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0265455-d1f4-499f-a391-42f45fd4ac30 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:41.612900 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0265455-d1f4-499f-a391-42f45fd4ac30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:41.613206 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0265455-d1f4-499f-a391-42f45fd4ac30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:41.614581 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0265455-d1f4-499f-a391-42f45fd4ac30 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:41.622492 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 601/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:27:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:41.628253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7797022-8ddb-4d09-a580-033f4adc4762 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:41.727066 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7797022-8ddb-4d09-a580-033f4adc4762 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:41.761203 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4f939-a5fd-4c6a-b446-df5dd961cdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:41.762049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4f939-a5fd-4c6a-b446-df5dd961cdca None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:41.762326 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4f939-a5fd-4c6a-b446-df5dd961cdca None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:41.762622 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4f939-a5fd-4c6a-b446-df5dd961cdca None admin] PATH_INFO: `/v3/users/5729cc57ecdf4f879ffafe0350ac808f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:41.776834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4f939-a5fd-4c6a-b446-df5dd961cdca None admin] RBAC: Authorizing `identity:get_user(user_id=5729cc57ecdf4f879ffafe0350ac808f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:41.778275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4f939-a5fd-4c6a-b446-df5dd961cdca None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:41.779921 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 581/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:27:41 2022] GET /identity/v3/users/5729cc57ecdf4f879ffafe0350ac808f => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:41.785687 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0265455-d1f4-499f-a391-42f45fd4ac30 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:41.933120 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0265455-d1f4-499f-a391-42f45fd4ac30 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:41.943063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:41.966718 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a97808-6da6-44f4-a720-340c965332cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:41.967348 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a97808-6da6-44f4-a720-340c965332cd None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:41.967626 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a97808-6da6-44f4-a720-340c965332cd None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:41.967903 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a97808-6da6-44f4-a720-340c965332cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:42.033333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a97808-6da6-44f4-a720-340c965332cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:42.034744 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a97808-6da6-44f4-a720-340c965332cd None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:42.128848 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 602/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:27:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:27:42.135265 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4f939-a5fd-4c6a-b446-df5dd961cdca None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:42.166147 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:42.234654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4f939-a5fd-4c6a-b446-df5dd961cdca None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:42.267991 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08776426-2d8a-449f-a030-29d013b1ef25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:42.268839 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08776426-2d8a-449f-a030-29d013b1ef25 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:42.269121 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08776426-2d8a-449f-a030-29d013b1ef25 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:42.269393 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08776426-2d8a-449f-a030-29d013b1ef25 None admin] PATH_INFO: `/v3/projects/a9769d7927534dae904ce8a3e272b7ff` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:42.270578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08776426-2d8a-449f-a030-29d013b1ef25 None admin] RBAC: Authorizing `identity:get_project(project_id=a9769d7927534dae904ce8a3e272b7ff)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:42.272020 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08776426-2d8a-449f-a030-29d013b1ef25 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:42.273690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 582/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:27:42 2022] GET /identity/v3/projects/a9769d7927534dae904ce8a3e272b7ff => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:42.279799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a97808-6da6-44f4-a720-340c965332cd None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:42.291109 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:42.376088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a97808-6da6-44f4-a720-340c965332cd None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:42.408425 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac35349-2beb-48f6-9632-5e28481c6a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:42.409058 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac35349-2beb-48f6-9632-5e28481c6a2f None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:42.409337 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac35349-2beb-48f6-9632-5e28481c6a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:42.409607 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac35349-2beb-48f6-9632-5e28481c6a2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:42.410422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac35349-2beb-48f6-9632-5e28481c6a2f None admin] RBAC: Adding query filter params (role.id=c9e006de0d6c4812ab29d778e9c8a830, scope.project.id=a9769d7927534dae904ce8a3e272b7ff, user.id=5729cc57ecdf4f879ffafe0350ac808f) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:42.410760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac35349-2beb-48f6-9632-5e28481c6a2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:42.412170 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac35349-2beb-48f6-9632-5e28481c6a2f None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:42.418417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 603/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:27:42 2022] GET /identity/v3/role_assignments?user.id=5729cc57ecdf4f879ffafe0350ac808f&scope.project.id=a9769d7927534dae904ce8a3e272b7ff&role.id=c9e006de0d6c4812ab29d778e9c8a830 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:44.415038 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068883d1-3fbb-49a3-b412-2eecba81a091 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:44.415572 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068883d1-3fbb-49a3-b412-2eecba81a091 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:44.415572 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068883d1-3fbb-49a3-b412-2eecba81a091 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:44.416937 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 583/1186] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:27:44.422792 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ed1f6-d2b2-4163-a6fc-8e12ee9873ca None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:44.423241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ed1f6-d2b2-4163-a6fc-8e12ee9873ca None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:44.423340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ed1f6-d2b2-4163-a6fc-8e12ee9873ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:44.439545 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-934ed1f6-d2b2-4163-a6fc-8e12ee9873ca None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:27:44.440706 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-934ed1f6-d2b2-4163-a6fc-8e12ee9873ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:44.506772 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 604/1187] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:27:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:27:44.529936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068883d1-3fbb-49a3-b412-2eecba81a091 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:44.624139 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:44.628593 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068883d1-3fbb-49a3-b412-2eecba81a091 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:44.660749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d4431-38b1-4011-928d-b2b126a72e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:44.661529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d4431-38b1-4011-928d-b2b126a72e58 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:44.661814 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d4431-38b1-4011-928d-b2b126a72e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:44.662088 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d4431-38b1-4011-928d-b2b126a72e58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:44.673083 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d4431-38b1-4011-928d-b2b126a72e58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:27:44.680776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-968d4431-38b1-4011-928d-b2b126a72e58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:27:44.680776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:27:44.680776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:27:44.680776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:27:44.680776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:27:44.680776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:27:44.680776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:27:44.680776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:27:44.680776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:27:44.680776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:27:44.680776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:27:44.680776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:27:44.680776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:27:44.680776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:27:44.680776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:27:44.680776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:27:44.681886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:27:44.681886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:27:44.681886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:27:44.681886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:27:44.681886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:27:44.681886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:27:44.681886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:27:44.681886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:27:44.681886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:27:44.681886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:27:44.681886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:27:44.681886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:27:44.681886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:27:44.681886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:27:44.681886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:27:44.681886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:27:44.682881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:27:44.682881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:27:44.682881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:27:44.682881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:27:44.682881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:27:44.682881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:27:44.682881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:27:44.682881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:27:44.682881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:27:44.682881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:27:44.682881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:27:44.682881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:27:44.682881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 584/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:27:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:27:44.687860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ed1f6-d2b2-4163-a6fc-8e12ee9873ca None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:44.690907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:44.783940 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ed1f6-d2b2-4163-a6fc-8e12ee9873ca None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:44.815890 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8889b5c-b8da-4664-adac-a6c873abd456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:44.816793 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8889b5c-b8da-4664-adac-a6c873abd456 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:44.817126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8889b5c-b8da-4664-adac-a6c873abd456 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:44.817398 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8889b5c-b8da-4664-adac-a6c873abd456 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:44.818139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8889b5c-b8da-4664-adac-a6c873abd456 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:44.818448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8889b5c-b8da-4664-adac-a6c873abd456 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:44.819836 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8889b5c-b8da-4664-adac-a6c873abd456 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:44.828364 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 605/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:27:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:44.834028 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d4431-38b1-4011-928d-b2b126a72e58 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:44.930370 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d4431-38b1-4011-928d-b2b126a72e58 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:44.963383 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b836e-901d-43b0-a06c-6e6f1901a685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:44.964231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b836e-901d-43b0-a06c-6e6f1901a685 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:44.964510 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b836e-901d-43b0-a06c-6e6f1901a685 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:44.964781 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b836e-901d-43b0-a06c-6e6f1901a685 None admin] PATH_INFO: `/v3/users/5729cc57ecdf4f879ffafe0350ac808f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:44.965944 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b836e-901d-43b0-a06c-6e6f1901a685 None admin] RBAC: Authorizing `identity:get_user(user_id=5729cc57ecdf4f879ffafe0350ac808f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:44.967395 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b836e-901d-43b0-a06c-6e6f1901a685 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:44.968993 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 585/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:27:44 2022] GET /identity/v3/users/5729cc57ecdf4f879ffafe0350ac808f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:44.974669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8889b5c-b8da-4664-adac-a6c873abd456 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:45.038066 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:45.072597 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8889b5c-b8da-4664-adac-a6c873abd456 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:45.109641 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0132cc-3990-4b8d-912d-0b65eebf04a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:45.111944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0132cc-3990-4b8d-912d-0b65eebf04a6 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:45.112229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0132cc-3990-4b8d-912d-0b65eebf04a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:45.112506 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0132cc-3990-4b8d-912d-0b65eebf04a6 None admin] PATH_INFO: `/v3/projects/a9769d7927534dae904ce8a3e272b7ff` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:45.113742 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0132cc-3990-4b8d-912d-0b65eebf04a6 None admin] RBAC: Authorizing `identity:get_project(project_id=a9769d7927534dae904ce8a3e272b7ff)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:45.115238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0132cc-3990-4b8d-912d-0b65eebf04a6 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:45.116948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 606/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:27:44 2022] GET /identity/v3/projects/a9769d7927534dae904ce8a3e272b7ff => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:45.123348 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b836e-901d-43b0-a06c-6e6f1901a685 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:45.151515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:45.222497 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b836e-901d-43b0-a06c-6e6f1901a685 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:45.255638 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163c697-a0b0-46a3-bf63-ae6344f521b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:45.256501 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163c697-a0b0-46a3-bf63-ae6344f521b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:45.256780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163c697-a0b0-46a3-bf63-ae6344f521b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:45.257050 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163c697-a0b0-46a3-bf63-ae6344f521b5 None admin] PATH_INFO: `/v3/projects/a9769d7927534dae904ce8a3e272b7ff/users/5729cc57ecdf4f879ffafe0350ac808f/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:45.260057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c163c697-a0b0-46a3-bf63-ae6344f521b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9769d7927534dae904ce8a3e272b7ff, user_id=5729cc57ecdf4f879ffafe0350ac808f, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:45.261474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c163c697-a0b0-46a3-bf63-ae6344f521b5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:45.326898 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 586/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:27:45 2022] PUT /identity/v3/projects/a9769d7927534dae904ce8a3e272b7ff/users/5729cc57ecdf4f879ffafe0350ac808f/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:27:47.315463 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800a006-bc2d-486a-8bff-dbcd66ddd67a None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:47.315990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800a006-bc2d-486a-8bff-dbcd66ddd67a None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:47.315990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800a006-bc2d-486a-8bff-dbcd66ddd67a None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:47.317612 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 607/1193] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:27:47.322875 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb090cd0-8af5-47d2-b787-0236f1e21d18 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:47.323263 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb090cd0-8af5-47d2-b787-0236f1e21d18 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:47.323427 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb090cd0-8af5-47d2-b787-0236f1e21d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:47.340207 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-eb090cd0-8af5-47d2-b787-0236f1e21d18 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:27:47.341367 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-eb090cd0-8af5-47d2-b787-0236f1e21d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:47.405433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 587/1194] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:27:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:27:47.428076 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800a006-bc2d-486a-8bff-dbcd66ddd67a None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:47.528728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800a006-bc2d-486a-8bff-dbcd66ddd67a None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:47.561162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed9ce84-d813-4b04-9d20-3e78225b0bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:47.561933 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9ce84-d813-4b04-9d20-3e78225b0bce None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:47.562216 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9ce84-d813-4b04-9d20-3e78225b0bce None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:47.562489 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9ce84-d813-4b04-9d20-3e78225b0bce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:47.564309 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:47.573702 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed9ce84-d813-4b04-9d20-3e78225b0bce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:27:47.588219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-5ed9ce84-d813-4b04-9d20-3e78225b0bce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:27:47.588219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:27:47.588219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:27:47.588219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:27:47.588219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:27:47.588219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:27:47.588219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:27:47.588219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:27:47.588219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:27:47.588219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:27:47.588219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:27:47.588219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:27:47.588219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:27:47.588219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:27:47.588219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:27:47.588219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:27:47.589486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:27:47.589486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:27:47.589486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:27:47.589486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:27:47.589486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:27:47.589486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:27:47.589486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:27:47.589486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:27:47.589486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:27:47.589486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:27:47.589486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:27:47.589486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:27:47.589486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:27:47.589486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:27:47.589486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:27:47.589486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:27:47.590475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:27:47.590475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:27:47.590475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:27:47.590475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:27:47.590475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:27:47.590475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:27:47.590475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:27:47.590475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:27:47.590475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:27:47.590475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:27:47.590475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:27:47.590475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:27:47.590475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 608/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:27:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:27:47.591235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb090cd0-8af5-47d2-b787-0236f1e21d18 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:47.605997 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:47.686704 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb090cd0-8af5-47d2-b787-0236f1e21d18 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:47.719082 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d6583-9059-4f66-9677-d11d6c1003ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:47.719736 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d6583-9059-4f66-9677-d11d6c1003ae None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:47.720017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d6583-9059-4f66-9677-d11d6c1003ae None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:47.720288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d6583-9059-4f66-9677-d11d6c1003ae None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:47.721021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d6583-9059-4f66-9677-d11d6c1003ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:47.721325 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d6583-9059-4f66-9677-d11d6c1003ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:47.722716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d6583-9059-4f66-9677-d11d6c1003ae None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:47.731018 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 588/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:27:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:47.736702 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed9ce84-d813-4b04-9d20-3e78225b0bce None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:47.832986 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed9ce84-d813-4b04-9d20-3e78225b0bce None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:47.864850 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82becdf3-5b08-40db-9159-9388550abdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:47.865698 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82becdf3-5b08-40db-9159-9388550abdbb None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:47.865983 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82becdf3-5b08-40db-9159-9388550abdbb None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:47.866255 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82becdf3-5b08-40db-9159-9388550abdbb None admin] PATH_INFO: `/v3/users/5729cc57ecdf4f879ffafe0350ac808f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:47.868131 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82becdf3-5b08-40db-9159-9388550abdbb None admin] RBAC: Authorizing `identity:get_user(user_id=5729cc57ecdf4f879ffafe0350ac808f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:47.869557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82becdf3-5b08-40db-9159-9388550abdbb None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:47.871192 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 609/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 23 11:27:47 2022] GET /identity/v3/users/5729cc57ecdf4f879ffafe0350ac808f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:47.877107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d6583-9059-4f66-9677-d11d6c1003ae None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:47.953736 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:47.974252 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d6583-9059-4f66-9677-d11d6c1003ae None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:48.007868 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f206e-0154-4621-9b77-6c49c1972b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:48.008766 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f206e-0154-4621-9b77-6c49c1972b86 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:48.009051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f206e-0154-4621-9b77-6c49c1972b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:48.009328 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f206e-0154-4621-9b77-6c49c1972b86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:48.074379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150f206e-0154-4621-9b77-6c49c1972b86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:48.075840 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150f206e-0154-4621-9b77-6c49c1972b86 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:48.150288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:48.172181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 589/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:27:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:27:48.177879 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82becdf3-5b08-40db-9159-9388550abdbb None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:48.237777 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:48.275393 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82becdf3-5b08-40db-9159-9388550abdbb None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:48.308005 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf77bf5-9b86-4a3d-8000-686c2cb2cfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:48.308846 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf77bf5-9b86-4a3d-8000-686c2cb2cfce None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:48.309130 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf77bf5-9b86-4a3d-8000-686c2cb2cfce None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:48.309404 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf77bf5-9b86-4a3d-8000-686c2cb2cfce None admin] PATH_INFO: `/v3/projects/a9769d7927534dae904ce8a3e272b7ff` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:48.310583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf77bf5-9b86-4a3d-8000-686c2cb2cfce None admin] RBAC: Authorizing `identity:get_project(project_id=a9769d7927534dae904ce8a3e272b7ff)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:48.312015 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf77bf5-9b86-4a3d-8000-686c2cb2cfce None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:48.313738 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 610/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 23 11:27:48 2022] GET /identity/v3/projects/a9769d7927534dae904ce8a3e272b7ff => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:48.320046 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f206e-0154-4621-9b77-6c49c1972b86 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:48.416207 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f206e-0154-4621-9b77-6c49c1972b86 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:48.449204 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43488f-678e-4991-9e18-ef8995b5f65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:48.449836 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43488f-678e-4991-9e18-ef8995b5f65b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:48.450115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43488f-678e-4991-9e18-ef8995b5f65b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:48.450437 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43488f-678e-4991-9e18-ef8995b5f65b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:48.451295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43488f-678e-4991-9e18-ef8995b5f65b None admin] RBAC: Adding query filter params (role.id=c9e006de0d6c4812ab29d778e9c8a830, scope.project.id=a9769d7927534dae904ce8a3e272b7ff, user.id=5729cc57ecdf4f879ffafe0350ac808f) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:27:48.451603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43488f-678e-4991-9e18-ef8995b5f65b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:48.453011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43488f-678e-4991-9e18-ef8995b5f65b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:48.458777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 590/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:27:48 2022] GET /identity/v3/role_assignments?user.id=5729cc57ecdf4f879ffafe0350ac808f&scope.project.id=a9769d7927534dae904ce8a3e272b7ff&role.id=c9e006de0d6c4812ab29d778e9c8a830 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:27:58.303866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=35 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:58.456181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=79 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:01.699015 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea6d2b-568e-4c52-b529-e46ddd65caa9 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:01.699607 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea6d2b-568e-4c52-b529-e46ddd65caa9 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:01.699607 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea6d2b-568e-4c52-b529-e46ddd65caa9 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:01.700945 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 611/1201] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:29:01.706334 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092f7b9-7a68-4420-ad1e-859f7e4ccb28 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:01.706767 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092f7b9-7a68-4420-ad1e-859f7e4ccb28 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:01.707110 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092f7b9-7a68-4420-ad1e-859f7e4ccb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:01.724009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d092f7b9-7a68-4420-ad1e-859f7e4ccb28 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:29:01.725165 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d092f7b9-7a68-4420-ad1e-859f7e4ccb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:29:01.793411 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 591/1202] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:29:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:29:01.818737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea6d2b-568e-4c52-b529-e46ddd65caa9 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:01.919243 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea6d2b-568e-4c52-b529-e46ddd65caa9 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:01.952453 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd68279-1cc3-4249-81e5-831b15ffb9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:01.953237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd68279-1cc3-4249-81e5-831b15ffb9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:01.953517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd68279-1cc3-4249-81e5-831b15ffb9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:01.953791 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd68279-1cc3-4249-81e5-831b15ffb9a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:01.963496 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd68279-1cc3-4249-81e5-831b15ffb9a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:01.965468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd68279-1cc3-4249-81e5-831b15ffb9a5 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:01.983600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-1fd68279-1cc3-4249-81e5-831b15ffb9a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:01.983600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:01.983600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:01.983600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:01.983600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:01.983600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:01.983600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:01.983600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:01.983600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:01.983600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:01.983600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:01.983600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:01.983600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:29:01.983600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:29:01.983600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:29:01.984648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:29:01.984648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:01.984648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:01.984648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:01.984648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:01.984648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:01.984648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:01.984648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:01.984648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:01.984648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:01.984648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:01.984648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:01.984648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:01.984648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:01.984648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:01.984648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:01.984648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:01.985713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:29:01.985713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:29:01.985713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:01.985713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:29:01.985713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 612/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:01.990898 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092f7b9-7a68-4420-ad1e-859f7e4ccb28 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:02.089712 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092f7b9-7a68-4420-ad1e-859f7e4ccb28 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:02.118662 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:02.124460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6a8db-adcb-469f-b9b9-bc23b14a59d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:02.127825 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6a8db-adcb-469f-b9b9-bc23b14a59d1 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:02.127825 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6a8db-adcb-469f-b9b9-bc23b14a59d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:02.127825 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6a8db-adcb-469f-b9b9-bc23b14a59d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:02.127825 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6a8db-adcb-469f-b9b9-bc23b14a59d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:02.127825 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6a8db-adcb-469f-b9b9-bc23b14a59d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:02.128227 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6a8db-adcb-469f-b9b9-bc23b14a59d1 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:02.138805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 592/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:02.144620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd68279-1cc3-4249-81e5-831b15ffb9a5 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:02.197550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:02.243466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd68279-1cc3-4249-81e5-831b15ffb9a5 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:02.276019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36817f4-083c-42af-bc54-809dc4aed0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:02.276693 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36817f4-083c-42af-bc54-809dc4aed0ad None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:02.277754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36817f4-083c-42af-bc54-809dc4aed0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:02.277958 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36817f4-083c-42af-bc54-809dc4aed0ad None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:02.278714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36817f4-083c-42af-bc54-809dc4aed0ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:02.280155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36817f4-083c-42af-bc54-809dc4aed0ad None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:02.382168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 613/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 23 11:29:02 2022] POST /identity/v3/users => generated 251 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:29:04.464734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a1ee3-ba1c-4903-95cb-bca3b8aafd30 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:04.465264 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a1ee3-ba1c-4903-95cb-bca3b8aafd30 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:04.465387 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a1ee3-ba1c-4903-95cb-bca3b8aafd30 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:04.466747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 593/1206] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:29:04.472314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bfe27-501d-44b5-aeec-4f4e9548ecd0 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:04.472707 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bfe27-501d-44b5-aeec-4f4e9548ecd0 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:04.472872 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bfe27-501d-44b5-aeec-4f4e9548ecd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:04.491960 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-b12bfe27-501d-44b5-aeec-4f4e9548ecd0 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:29:04.494164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b12bfe27-501d-44b5-aeec-4f4e9548ecd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:29:04.559949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 614/1207] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:29:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:29:04.584347 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a1ee3-ba1c-4903-95cb-bca3b8aafd30 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:04.686499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a1ee3-ba1c-4903-95cb-bca3b8aafd30 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:04.719687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad7cd1-c76c-4da3-a7a2-10c15c6eff53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:04.720408 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad7cd1-c76c-4da3-a7a2-10c15c6eff53 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:04.720685 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad7cd1-c76c-4da3-a7a2-10c15c6eff53 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:04.720952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad7cd1-c76c-4da3-a7a2-10c15c6eff53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:04.731698 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad7cd1-c76c-4da3-a7a2-10c15c6eff53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:29:04.739366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-91ad7cd1-c76c-4da3-a7a2-10c15c6eff53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:29:04.739366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:04.739366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:04.739366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:04.739366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:04.739366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:04.739366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:04.739366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:04.739366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:04.739366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:04.739366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:04.739366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:04.739366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:29:04.739366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:29:04.739366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:29:04.739366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:29:04.740376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:29:04.740376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:29:04.740376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:29:04.740376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:29:04.740376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:04.740376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:04.740376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:04.740376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:04.740376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:04.740376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:04.740376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:04.740376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:04.740376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:04.740376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:04.740376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:04.740376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:04.741351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:04.741351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:04.741351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:04.741351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:04.741351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:29:04.741351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:29:04.741351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:29:04.741351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:29:04.741351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:29:04.741351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:29:04.741351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:29:04.741351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:04.741351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 594/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:29:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:04.747456 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bfe27-501d-44b5-aeec-4f4e9548ecd0 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:04.786818 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:04.846260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bfe27-501d-44b5-aeec-4f4e9548ecd0 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:04.878974 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffeab1a-1db3-40f1-b491-d495d44f36ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:04.879631 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffeab1a-1db3-40f1-b491-d495d44f36ad None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:04.879915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffeab1a-1db3-40f1-b491-d495d44f36ad None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:04.880192 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffeab1a-1db3-40f1-b491-d495d44f36ad None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:04.880930 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffeab1a-1db3-40f1-b491-d495d44f36ad None admin] RBAC: Adding query filter params (name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:04.881235 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffeab1a-1db3-40f1-b491-d495d44f36ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:04.882680 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffeab1a-1db3-40f1-b491-d495d44f36ad None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:04.891663 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 615/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:29:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:04.899146 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad7cd1-c76c-4da3-a7a2-10c15c6eff53 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:04.947269 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:04.996361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad7cd1-c76c-4da3-a7a2-10c15c6eff53 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:05.029854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca83da-3cae-4d17-9466-785323c5205d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:05.030659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca83da-3cae-4d17-9466-785323c5205d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:05.030945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca83da-3cae-4d17-9466-785323c5205d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:05.031319 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca83da-3cae-4d17-9466-785323c5205d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:05.041615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca83da-3cae-4d17-9466-785323c5205d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:05.043978 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca83da-3cae-4d17-9466-785323c5205d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:05.060888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-d9ca83da-3cae-4d17-9466-785323c5205d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:05.060888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:05.060888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:05.060888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:05.060888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:05.060888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:05.060888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:05.060888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:05.060888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:05.060888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:05.060888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:05.060888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:05.060888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:29:05.060888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:29:05.060888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:29:05.062056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:29:05.062056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:05.062056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:05.062056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:05.062056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:05.062056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:05.062056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:05.062056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:05.062056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:05.062056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:05.062056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:05.062056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:05.062056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:05.062056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:05.062056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:05.062056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:05.062056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:05.063161 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:29:05.063161 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:29:05.063161 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:05.063161 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:05.063161 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 595/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:29:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:05.068078 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffeab1a-1db3-40f1-b491-d495d44f36ad None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:05.165032 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:05.168600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffeab1a-1db3-40f1-b491-d495d44f36ad None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:05.201955 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6e8a3-85af-4846-ac4d-62999e427b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:05.202709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6e8a3-85af-4846-ac4d-62999e427b14 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:05.202997 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6e8a3-85af-4846-ac4d-62999e427b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:05.203273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6e8a3-85af-4846-ac4d-62999e427b14 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:05.203994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6e8a3-85af-4846-ac4d-62999e427b14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:05.204303 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6e8a3-85af-4846-ac4d-62999e427b14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:05.205752 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6e8a3-85af-4846-ac4d-62999e427b14 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:05.217434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 616/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:05.223061 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca83da-3cae-4d17-9466-785323c5205d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:05.346894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca83da-3cae-4d17-9466-785323c5205d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:05.346894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:05.362400 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538b198-5937-48e6-b985-d06ae9937545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:05.363220 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538b198-5937-48e6-b985-d06ae9937545 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:05.363500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538b198-5937-48e6-b985-d06ae9937545 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:05.363817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538b198-5937-48e6-b985-d06ae9937545 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:05.371049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538b198-5937-48e6-b985-d06ae9937545 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:05.372483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538b198-5937-48e6-b985-d06ae9937545 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:05.387824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-b538b198-5937-48e6-b985-d06ae9937545 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:29:05.387824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:05.387824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:05.387824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:05.387824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:05.387824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:05.387824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:05.387824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:05.387824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:05.387824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:05.387824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:05.387824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:05.387824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:29:05.387824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:29:05.387824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:29:05.387824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:29:05.388995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:05.388995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:05.388995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:29:05.388995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:29:05.388995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:29:05.388995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:29:05.388995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:05.388995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:05.388995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:05.388995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:05.388995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:05.388995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:05.388995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:05.388995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:05.388995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:05.388995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:05.388995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:05.388995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:05.390238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:05.390238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:05.390238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:29:05.390238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:29:05.390238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:29:05.390238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:29:05.390238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:29:05.390238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:29:05.390238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:29:05.390238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:05.390238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 596/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:29:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:05.394875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6e8a3-85af-4846-ac4d-62999e427b14 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:05.494559 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6e8a3-85af-4846-ac4d-62999e427b14 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:05.528462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4292ff-b941-4c87-9452-95a3089851d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:05.529131 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4292ff-b941-4c87-9452-95a3089851d9 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:05.529416 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4292ff-b941-4c87-9452-95a3089851d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:05.529693 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4292ff-b941-4c87-9452-95a3089851d9 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:05.530681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4292ff-b941-4c87-9452-95a3089851d9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:05.530996 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4292ff-b941-4c87-9452-95a3089851d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:05.532416 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4292ff-b941-4c87-9452-95a3089851d9 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:05.549184 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 617/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Jun 23 11:29:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:05.554955 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538b198-5937-48e6-b985-d06ae9937545 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:05.653187 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538b198-5937-48e6-b985-d06ae9937545 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:05.687657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542cdf6-ea5b-4716-8ee3-a20e8c5094f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:05.688293 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542cdf6-ea5b-4716-8ee3-a20e8c5094f5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:05.688570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542cdf6-ea5b-4716-8ee3-a20e8c5094f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:05.688838 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542cdf6-ea5b-4716-8ee3-a20e8c5094f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:05.719130 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:05.756879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542cdf6-ea5b-4716-8ee3-a20e8c5094f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:05.758402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542cdf6-ea5b-4716-8ee3-a20e8c5094f5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:05.854758 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 597/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:29:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:29:05.860567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4292ff-b941-4c87-9452-95a3089851d9 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:05.879608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:05.959641 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4292ff-b941-4c87-9452-95a3089851d9 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:05.993312 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b16a7b-d180-42a4-a6ca-1960a5516572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:05.994095 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b16a7b-d180-42a4-a6ca-1960a5516572 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:05.994387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b16a7b-d180-42a4-a6ca-1960a5516572 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:05.994693 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b16a7b-d180-42a4-a6ca-1960a5516572 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:06.005124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b16a7b-d180-42a4-a6ca-1960a5516572 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:06.006598 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b16a7b-d180-42a4-a6ca-1960a5516572 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:06.023613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-c5b16a7b-d180-42a4-a6ca-1960a5516572 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:06.023613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:06.023613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:06.023613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:06.023613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:06.023613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:06.023613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:06.023613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:06.023613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:06.023613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:06.023613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:06.023613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:06.023613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:29:06.023613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:29:06.023613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:29:06.024704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:29:06.024704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:06.024704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:06.024704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:06.024704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:06.024704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:06.024704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:06.024704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:06.024704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:06.024704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:06.024704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:06.024704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:06.024704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:06.024704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:06.024704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:06.024704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:06.024704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:06.036193 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:29:06.036193 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:29:06.036193 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:06.036193 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:29:06.036193 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 618/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:06.036518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542cdf6-ea5b-4716-8ee3-a20e8c5094f5 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:06.090794 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:06.128572 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542cdf6-ea5b-4716-8ee3-a20e8c5094f5 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:06.160391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade5be5-0521-4b1d-97d5-193cd52e7003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:06.164164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade5be5-0521-4b1d-97d5-193cd52e7003 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:06.164164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade5be5-0521-4b1d-97d5-193cd52e7003 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:06.164164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade5be5-0521-4b1d-97d5-193cd52e7003 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:06.164164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade5be5-0521-4b1d-97d5-193cd52e7003 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:06.164164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade5be5-0521-4b1d-97d5-193cd52e7003 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:06.164164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade5be5-0521-4b1d-97d5-193cd52e7003 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:06.176021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 598/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:06.181606 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b16a7b-d180-42a4-a6ca-1960a5516572 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:06.250138 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:06.279632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b16a7b-d180-42a4-a6ca-1960a5516572 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:06.312205 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4984f0f-bfaa-409b-8f95-3897a375c1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:06.312983 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4984f0f-bfaa-409b-8f95-3897a375c1db None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:06.313304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4984f0f-bfaa-409b-8f95-3897a375c1db None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:06.313595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4984f0f-bfaa-409b-8f95-3897a375c1db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:06.320243 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4984f0f-bfaa-409b-8f95-3897a375c1db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:06.321674 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4984f0f-bfaa-409b-8f95-3897a375c1db None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:06.335513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-e4984f0f-bfaa-409b-8f95-3897a375c1db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:29:06.335513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:06.335513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:06.335513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:06.335513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:06.335513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:06.335513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:06.335513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:06.335513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:06.335513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:06.335513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:06.335513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:06.335513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:29:06.335513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:29:06.335513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:29:06.336688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:29:06.336688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:06.336688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:06.336688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:06.336688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:06.336688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:06.336688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:06.336688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:06.336688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:06.336688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:06.336688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:06.336688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:06.336688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:06.336688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:06.336688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:06.336688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:06.336688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:06.337789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:29:06.337789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:29:06.337789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:29:06.337789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:29:06.337789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:29:06.337789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:29:06.337789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:29:06.337789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:29:06.337789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 619/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:29:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:06.343124 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade5be5-0521-4b1d-97d5-193cd52e7003 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:06.440616 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade5be5-0521-4b1d-97d5-193cd52e7003 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:06.462727 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:06.474284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d43124-7591-4bd9-b62a-a8fc67560394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:06.474968 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d43124-7591-4bd9-b62a-a8fc67560394 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:06.475246 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d43124-7591-4bd9-b62a-a8fc67560394 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:06.475514 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d43124-7591-4bd9-b62a-a8fc67560394 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:06.476283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d43124-7591-4bd9-b62a-a8fc67560394 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:06.476592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d43124-7591-4bd9-b62a-a8fc67560394 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:06.478010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d43124-7591-4bd9-b62a-a8fc67560394 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:06.490385 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 599/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:29:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:06.496267 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4984f0f-bfaa-409b-8f95-3897a375c1db None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:06.592166 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4984f0f-bfaa-409b-8f95-3897a375c1db None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:06.620209 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:06.626262 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ccade-6318-448a-b5a6-a2dc945d3831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:06.626951 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ccade-6318-448a-b5a6-a2dc945d3831 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:06.627231 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ccade-6318-448a-b5a6-a2dc945d3831 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:06.627502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ccade-6318-448a-b5a6-a2dc945d3831 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:06.628314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ccade-6318-448a-b5a6-a2dc945d3831 None admin] RBAC: Adding query filter params (role.id=13708a4819a94b94b47ba5994aaa9310, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=ac04b32777ca4f929e9e59deee6e0260) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:06.628684 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ccade-6318-448a-b5a6-a2dc945d3831 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:06.630066 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ccade-6318-448a-b5a6-a2dc945d3831 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:06.635305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 620/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:29:06 2022] GET /identity/v3/role_assignments?user.id=ac04b32777ca4f929e9e59deee6e0260&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=13708a4819a94b94b47ba5994aaa9310 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:29:08.646098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc7ca8-ad2a-4996-85c8-b6a59405b310 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:08.646700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc7ca8-ad2a-4996-85c8-b6a59405b310 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:08.646700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc7ca8-ad2a-4996-85c8-b6a59405b310 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:08.648050 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 600/1220] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:29:08.653994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9b068-c45f-4e25-9634-e66cc1e3a9ba None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:08.657188 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9b068-c45f-4e25-9634-e66cc1e3a9ba None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:08.657188 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9b068-c45f-4e25-9634-e66cc1e3a9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:08.671797 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-fab9b068-c45f-4e25-9634-e66cc1e3a9ba None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:29:08.672982 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-fab9b068-c45f-4e25-9634-e66cc1e3a9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:29:08.738953 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 621/1221] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:29:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:29:08.762645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc7ca8-ad2a-4996-85c8-b6a59405b310 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:08.878780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc7ca8-ad2a-4996-85c8-b6a59405b310 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:08.911903 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99b48e-53bf-4c77-a2b8-de66ca7e7c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:08.912730 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99b48e-53bf-4c77-a2b8-de66ca7e7c77 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:08.913015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99b48e-53bf-4c77-a2b8-de66ca7e7c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:08.913290 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99b48e-53bf-4c77-a2b8-de66ca7e7c77 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:08.924055 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99b48e-53bf-4c77-a2b8-de66ca7e7c77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:29:08.931811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-7c99b48e-53bf-4c77-a2b8-de66ca7e7c77 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:29:08.931811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:08.931811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:08.931811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:08.931811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:08.931811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:08.931811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:08.931811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:08.931811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:08.931811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:08.931811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:08.931811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:08.931811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:29:08.931811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:29:08.931811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:29:08.931811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:29:08.933503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:29:08.933503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:29:08.933503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:29:08.933503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:29:08.933503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:08.933503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:08.933503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:08.933503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:08.933503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:08.933503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:08.933503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:08.933503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:08.933503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:08.933503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:08.933503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:08.933503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:08.934706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:08.934706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:08.934706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:08.934706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:08.934706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:29:08.934706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:29:08.934706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:29:08.934706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:29:08.934706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:29:08.934706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:29:08.934706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:29:08.934706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:08.934706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 601/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:29:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:29:08.939287 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9b068-c45f-4e25-9634-e66cc1e3a9ba None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:09.035673 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9b068-c45f-4e25-9634-e66cc1e3a9ba None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:09.058144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:09.068611 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50165473-3dba-4167-90e3-60e691015bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:09.069267 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50165473-3dba-4167-90e3-60e691015bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:09.069551 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50165473-3dba-4167-90e3-60e691015bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:09.069828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50165473-3dba-4167-90e3-60e691015bd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:09.070592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50165473-3dba-4167-90e3-60e691015bd4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:09.070905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50165473-3dba-4167-90e3-60e691015bd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:09.072264 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50165473-3dba-4167-90e3-60e691015bd4 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:09.080622 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 622/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:29:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:09.086233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99b48e-53bf-4c77-a2b8-de66ca7e7c77 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:09.131521 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:09.187632 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99b48e-53bf-4c77-a2b8-de66ca7e7c77 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:09.220979 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4422bdb-c501-49ba-ab5b-a48d1dcb23be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:09.221764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4422bdb-c501-49ba-ab5b-a48d1dcb23be None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:09.222096 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4422bdb-c501-49ba-ab5b-a48d1dcb23be None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:09.222329 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4422bdb-c501-49ba-ab5b-a48d1dcb23be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:09.233310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4422bdb-c501-49ba-ab5b-a48d1dcb23be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:09.234777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4422bdb-c501-49ba-ab5b-a48d1dcb23be None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:09.251906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-a4422bdb-c501-49ba-ab5b-a48d1dcb23be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:09.251906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:09.251906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:09.251906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:09.251906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:09.251906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:09.251906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:09.251906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:09.251906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:09.251906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:09.251906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:09.251906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:09.251906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:29:09.251906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:29:09.251906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:29:09.252981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:29:09.252981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:09.252981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:09.252981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:09.252981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:09.252981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:09.252981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:09.252981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:09.252981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:09.252981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:09.252981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:09.252981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:09.252981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:09.252981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:09.252981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:09.252981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:09.252981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:09.253896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:29:09.253896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:29:09.253896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:09.253896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:09.253896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 602/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:09.258937 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50165473-3dba-4167-90e3-60e691015bd4 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:09.360744 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50165473-3dba-4167-90e3-60e691015bd4 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:09.392611 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae03a4-b6cd-45d5-9f13-4b93992ea836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:09.393257 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae03a4-b6cd-45d5-9f13-4b93992ea836 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:09.393539 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae03a4-b6cd-45d5-9f13-4b93992ea836 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:09.393814 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae03a4-b6cd-45d5-9f13-4b93992ea836 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:09.394548 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae03a4-b6cd-45d5-9f13-4b93992ea836 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:09.394968 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae03a4-b6cd-45d5-9f13-4b93992ea836 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:09.396331 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae03a4-b6cd-45d5-9f13-4b93992ea836 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:09.408376 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 623/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:09.413998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4422bdb-c501-49ba-ab5b-a48d1dcb23be None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:09.507791 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:09.514388 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4422bdb-c501-49ba-ab5b-a48d1dcb23be None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:09.547952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e555da4-76d9-4159-bb83-9d62f2aaafe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:09.548743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e555da4-76d9-4159-bb83-9d62f2aaafe0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:09.549024 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e555da4-76d9-4159-bb83-9d62f2aaafe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:09.549297 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e555da4-76d9-4159-bb83-9d62f2aaafe0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:09.556201 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e555da4-76d9-4159-bb83-9d62f2aaafe0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:09.557701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e555da4-76d9-4159-bb83-9d62f2aaafe0 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:09.572745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-9e555da4-76d9-4159-bb83-9d62f2aaafe0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:29:09.572745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:09.572745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:09.572745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:09.572745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:09.572745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:09.572745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:09.572745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:09.572745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:09.572745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:09.572745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:09.572745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:09.572745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:29:09.572745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:29:09.572745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:29:09.572745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:29:09.573925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:09.573925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:09.573925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:29:09.573925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:29:09.573925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:29:09.573925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:29:09.573925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:09.573925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:09.573925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:09.573925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:09.573925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:09.573925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:09.573925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:09.573925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:09.573925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:09.573925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:09.573925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:09.573925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:09.575181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:09.575181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:09.575181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:29:09.575181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:29:09.575181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:29:09.575181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:29:09.575181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:29:09.575181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:29:09.575181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:29:09.575181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:09.575181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 603/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:29:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:09.580025 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae03a4-b6cd-45d5-9f13-4b93992ea836 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:09.627145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:09.679030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae03a4-b6cd-45d5-9f13-4b93992ea836 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:09.712036 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c3fef5-831b-4e38-800d-42236780abc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:09.712692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c3fef5-831b-4e38-800d-42236780abc1 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:09.712977 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c3fef5-831b-4e38-800d-42236780abc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:09.713254 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c3fef5-831b-4e38-800d-42236780abc1 None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:09.714979 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c3fef5-831b-4e38-800d-42236780abc1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:09.715113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c3fef5-831b-4e38-800d-42236780abc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:09.716482 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c3fef5-831b-4e38-800d-42236780abc1 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:09.733596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 624/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Jun 23 11:29:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:09.738996 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e555da4-76d9-4159-bb83-9d62f2aaafe0 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:09.838820 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e555da4-76d9-4159-bb83-9d62f2aaafe0 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:09.873632 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e4a66-ed31-449a-98fc-8e66a359fc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:09.874421 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e4a66-ed31-449a-98fc-8e66a359fc21 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:09.874727 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e4a66-ed31-449a-98fc-8e66a359fc21 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:09.875004 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e4a66-ed31-449a-98fc-8e66a359fc21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:09.885214 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e4a66-ed31-449a-98fc-8e66a359fc21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:09.886808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e4a66-ed31-449a-98fc-8e66a359fc21 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:09.904805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-116e4a66-ed31-449a-98fc-8e66a359fc21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:09.904805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:09.904805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:09.904805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:09.904805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:09.904805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:09.904805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:09.904805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:09.904805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:09.904805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:09.904805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:09.904805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:09.904805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:29:09.904805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:29:09.904805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:29:09.905889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:29:09.905889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:09.905889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:09.905889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:09.905889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:09.905889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:09.905889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:09.905889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:09.905889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:09.905889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:09.905889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:09.905889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:09.905889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:09.905889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:09.905889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:09.905889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:09.905889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:09.907389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:29:09.907389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:29:09.907389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:09.907389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:09.907389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 604/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:09.911705 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c3fef5-831b-4e38-800d-42236780abc1 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:10.011045 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c3fef5-831b-4e38-800d-42236780abc1 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:10.013276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:10.045545 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2419735-4052-41e8-9bb6-6db026915993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:10.046189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2419735-4052-41e8-9bb6-6db026915993 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:10.046467 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2419735-4052-41e8-9bb6-6db026915993 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:10.046769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2419735-4052-41e8-9bb6-6db026915993 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:10.047478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2419735-4052-41e8-9bb6-6db026915993 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:10.047780 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2419735-4052-41e8-9bb6-6db026915993 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:10.049306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2419735-4052-41e8-9bb6-6db026915993 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:10.060233 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 625/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:10.066336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e4a66-ed31-449a-98fc-8e66a359fc21 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:10.075249 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:10.163988 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e4a66-ed31-449a-98fc-8e66a359fc21 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:10.204099 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b0da2-1a89-4fee-acb3-305d5a8474bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:10.204915 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b0da2-1a89-4fee-acb3-305d5a8474bb None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:10.205199 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b0da2-1a89-4fee-acb3-305d5a8474bb None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:10.205475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b0da2-1a89-4fee-acb3-305d5a8474bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:10.211989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b0da2-1a89-4fee-acb3-305d5a8474bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:10.213430 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b0da2-1a89-4fee-acb3-305d5a8474bb None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:10.227169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-889b0da2-1a89-4fee-acb3-305d5a8474bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:29:10.227169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:10.227169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:10.227169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:10.227169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:10.227169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:10.227169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:10.227169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:10.227169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:10.227169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:10.227169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:10.227169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:10.227169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:29:10.227169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:29:10.227169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:29:10.228502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:29:10.228502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:10.228502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:10.228502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:10.228502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:10.228502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:10.228502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:10.228502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:10.228502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:10.228502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:10.228502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:10.228502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:10.228502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:10.228502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:10.228502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:10.228502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:10.228502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:10.229475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:29:10.229475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:29:10.229475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:29:10.229475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:29:10.229475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:29:10.229475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:29:10.229475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:29:10.229475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:10.229475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 605/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:29:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:10.235061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2419735-4052-41e8-9bb6-6db026915993 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:10.332877 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2419735-4052-41e8-9bb6-6db026915993 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:10.366614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8fcc4-e4a2-4999-8e94-388d3fd6b3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:10.367269 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8fcc4-e4a2-4999-8e94-388d3fd6b3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:10.367561 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8fcc4-e4a2-4999-8e94-388d3fd6b3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:10.367839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8fcc4-e4a2-4999-8e94-388d3fd6b3b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:10.368619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8fcc4-e4a2-4999-8e94-388d3fd6b3b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:10.368930 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8fcc4-e4a2-4999-8e94-388d3fd6b3b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:10.370450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8fcc4-e4a2-4999-8e94-388d3fd6b3b9 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:10.382255 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 626/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:29:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:10.388445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b0da2-1a89-4fee-acb3-305d5a8474bb None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:10.456708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:10.492083 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b0da2-1a89-4fee-acb3-305d5a8474bb None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:10.528142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c593ce-4744-4214-86e4-6d663b71187b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:10.529023 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c593ce-4744-4214-86e4-6d663b71187b None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:10.529300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c593ce-4744-4214-86e4-6d663b71187b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:10.529571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c593ce-4744-4214-86e4-6d663b71187b None admin] PATH_INFO: `/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/ac04b32777ca4f929e9e59deee6e0260/roles/13708a4819a94b94b47ba5994aaa9310` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:10.544030 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c593ce-4744-4214-86e4-6d663b71187b None admin] RBAC: Authorizing `identity:create_grant(project_id=886c2939a14b48febc2e2fe639c8f1e4, user_id=ac04b32777ca4f929e9e59deee6e0260, role_id=13708a4819a94b94b47ba5994aaa9310)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:10.545484 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c593ce-4744-4214-86e4-6d663b71187b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:10.684402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 606/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:29:10 2022] PUT /identity/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/ac04b32777ca4f929e9e59deee6e0260/roles/13708a4819a94b94b47ba5994aaa9310 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:29:12.719786 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d863f51-2e09-4125-bb37-40e7a6c73dfb None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:12.720336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d863f51-2e09-4125-bb37-40e7a6c73dfb None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:12.720427 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d863f51-2e09-4125-bb37-40e7a6c73dfb None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:12.721772 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 627/1233] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:29:12.727091 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eb6fb-bbfb-4888-9b80-eb13cac23918 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:12.727507 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eb6fb-bbfb-4888-9b80-eb13cac23918 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:12.727660 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eb6fb-bbfb-4888-9b80-eb13cac23918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:12.744753 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-260eb6fb-bbfb-4888-9b80-eb13cac23918 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:29:12.746386 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-260eb6fb-bbfb-4888-9b80-eb13cac23918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:29:12.813119 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 607/1234] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:29:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:29:12.837528 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d863f51-2e09-4125-bb37-40e7a6c73dfb None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:12.863334 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:12.939317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d863f51-2e09-4125-bb37-40e7a6c73dfb None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:12.974011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288bf60-f48f-407a-856f-6bfc2703dc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:12.974806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288bf60-f48f-407a-856f-6bfc2703dc4b None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:12.975088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288bf60-f48f-407a-856f-6bfc2703dc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:12.975436 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288bf60-f48f-407a-856f-6bfc2703dc4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:12.985730 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288bf60-f48f-407a-856f-6bfc2703dc4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:29:12.993371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-2288bf60-f48f-407a-856f-6bfc2703dc4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:29:12.993371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:12.993371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:12.993371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:12.993371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:12.993371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:12.993371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:12.993371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:12.993371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:12.993371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:12.993371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:12.993371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:12.993371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:29:12.993371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:29:12.993371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:29:12.993371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:29:12.994479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:29:12.994479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:29:12.994479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:29:12.994479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:29:12.994479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:12.994479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:12.994479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:12.994479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:12.994479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:12.994479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:12.994479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:12.994479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:12.994479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:12.994479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:12.994479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:12.994479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:12.995645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:12.995645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:12.995645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:12.995645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:12.995645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:29:12.995645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:29:12.995645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:29:12.995645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:29:12.995645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:29:12.995645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:29:12.995645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:29:12.995645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:29:12.995645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 628/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 23 11:29:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:13.001092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260eb6fb-bbfb-4888-9b80-eb13cac23918 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:13.048411 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:13.102440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260eb6fb-bbfb-4888-9b80-eb13cac23918 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:13.136346 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b11cf-0032-413e-a604-87c128a2aea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:13.137009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b11cf-0032-413e-a604-87c128a2aea3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:13.137289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b11cf-0032-413e-a604-87c128a2aea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:13.137563 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b11cf-0032-413e-a604-87c128a2aea3 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:13.138303 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b11cf-0032-413e-a604-87c128a2aea3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:13.138640 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b11cf-0032-413e-a604-87c128a2aea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:13.139998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b11cf-0032-413e-a604-87c128a2aea3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:13.148194 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 608/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 23 11:29:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:13.153767 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288bf60-f48f-407a-856f-6bfc2703dc4b None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:13.237280 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:13.252912 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288bf60-f48f-407a-856f-6bfc2703dc4b None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:13.286118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5c11f-b98d-4f27-bb42-a3224b933a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:13.286925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5c11f-b98d-4f27-bb42-a3224b933a39 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:13.287208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5c11f-b98d-4f27-bb42-a3224b933a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:13.287479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5c11f-b98d-4f27-bb42-a3224b933a39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:13.297238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c5c11f-b98d-4f27-bb42-a3224b933a39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:13.298717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c5c11f-b98d-4f27-bb42-a3224b933a39 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:13.316305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-a9c5c11f-b98d-4f27-bb42-a3224b933a39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:13.316305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:13.316305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:13.316305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:13.316305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:13.316305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:13.316305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:13.316305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:13.316305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:13.316305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:13.316305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:13.316305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:13.316305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:29:13.316305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:29:13.316305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:29:13.317363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:29:13.317363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:13.317363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:13.317363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:13.317363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:13.317363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:13.317363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:13.317363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:13.317363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:13.317363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:13.317363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:13.317363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:13.317363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:13.317363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:13.317363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:13.317363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:13.317363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:13.318335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:29:13.318335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:29:13.318335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:13.318335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:29:13.318335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 629/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:29:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:13.324264 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b11cf-0032-413e-a604-87c128a2aea3 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:13.422915 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b11cf-0032-413e-a604-87c128a2aea3 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:13.431484 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:13.455846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b7d1c-7e53-4b2f-9f37-2f0279a750f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:13.456500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b7d1c-7e53-4b2f-9f37-2f0279a750f1 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:13.456782 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b7d1c-7e53-4b2f-9f37-2f0279a750f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:13.457059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b7d1c-7e53-4b2f-9f37-2f0279a750f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:13.457772 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b7d1c-7e53-4b2f-9f37-2f0279a750f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:13.458079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b7d1c-7e53-4b2f-9f37-2f0279a750f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:13.459498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b7d1c-7e53-4b2f-9f37-2f0279a750f1 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:13.470906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 609/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:13.476532 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5c11f-b98d-4f27-bb42-a3224b933a39 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:13.581575 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5c11f-b98d-4f27-bb42-a3224b933a39 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:13.616044 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd1e80-3527-476a-9774-5f32def96dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:13.616841 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd1e80-3527-476a-9774-5f32def96dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:13.617126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd1e80-3527-476a-9774-5f32def96dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:13.617419 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd1e80-3527-476a-9774-5f32def96dd5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:13.620665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:13.625488 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd1e80-3527-476a-9774-5f32def96dd5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:13.627029 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd1e80-3527-476a-9774-5f32def96dd5 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:13.642754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-2fcd1e80-3527-476a-9774-5f32def96dd5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:29:13.642754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:13.642754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:13.642754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:13.642754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:13.642754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:13.642754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:13.642754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:13.642754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:13.642754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:13.642754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:13.642754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:13.642754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:29:13.642754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:29:13.642754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:29:13.642754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:29:13.643931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:13.643931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:13.643931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:29:13.643931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:29:13.643931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:29:13.643931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:29:13.643931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:13.643931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:13.643931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:13.643931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:13.643931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:13.643931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:13.643931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:13.643931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:13.643931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:13.643931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:13.643931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:13.643931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:13.644915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:13.644915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:13.644915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:29:13.644915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:29:13.644915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:29:13.644915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:29:13.644915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:29:13.644915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:29:13.644915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:29:13.644915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:29:13.644915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 630/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:29:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:13.649720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b7d1c-7e53-4b2f-9f37-2f0279a750f1 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:13.746968 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b7d1c-7e53-4b2f-9f37-2f0279a750f1 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:13.782582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3609b8-ab7d-46f9-abd8-ef46c39cfc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:13.783289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3609b8-ab7d-46f9-abd8-ef46c39cfc1d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:13.783566 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3609b8-ab7d-46f9-abd8-ef46c39cfc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:13.783838 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3609b8-ab7d-46f9-abd8-ef46c39cfc1d None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:13.784671 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3609b8-ab7d-46f9-abd8-ef46c39cfc1d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:13.784976 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3609b8-ab7d-46f9-abd8-ef46c39cfc1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:13.786398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3609b8-ab7d-46f9-abd8-ef46c39cfc1d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:13.804536 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 610/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Jun 23 11:29:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:13.810436 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd1e80-3527-476a-9774-5f32def96dd5 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:13.908510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd1e80-3527-476a-9774-5f32def96dd5 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:13.943525 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797e8ea-fce4-48e9-a306-26d8715a88bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:13.944277 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797e8ea-fce4-48e9-a306-26d8715a88bf None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:13.944556 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797e8ea-fce4-48e9-a306-26d8715a88bf None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:13.944831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797e8ea-fce4-48e9-a306-26d8715a88bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:14.004398 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:14.011020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797e8ea-fce4-48e9-a306-26d8715a88bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:14.012516 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797e8ea-fce4-48e9-a306-26d8715a88bf None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:14.108661 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 631/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:29:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:29:14.114806 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3609b8-ab7d-46f9-abd8-ef46c39cfc1d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:14.142412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:14.212794 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3609b8-ab7d-46f9-abd8-ef46c39cfc1d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:14.246447 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34988dc6-8dec-4211-b7b4-0384e20c931c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:14.247258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34988dc6-8dec-4211-b7b4-0384e20c931c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:14.247541 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34988dc6-8dec-4211-b7b4-0384e20c931c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:14.247817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34988dc6-8dec-4211-b7b4-0384e20c931c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:14.257567 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34988dc6-8dec-4211-b7b4-0384e20c931c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:14.259062 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34988dc6-8dec-4211-b7b4-0384e20c931c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:14.277179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-34988dc6-8dec-4211-b7b4-0384e20c931c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:14.277179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:14.277179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:14.277179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:14.277179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:14.277179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:14.277179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:14.277179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:14.277179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:14.277179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:14.277179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:14.277179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:14.277179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:29:14.277179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:29:14.277179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:29:14.278300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:29:14.278300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:14.278300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:14.278300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:14.278300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:14.278300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:14.278300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:14.278300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:14.278300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:14.278300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:14.278300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:14.278300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:14.278300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:14.278300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:14.278300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:14.278300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:14.278300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:14.279475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:29:14.279475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:29:14.279475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:14.279475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:14.279475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 611/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:29:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:14.284245 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797e8ea-fce4-48e9-a306-26d8715a88bf None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:14.374826 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:14.380373 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797e8ea-fce4-48e9-a306-26d8715a88bf None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:14.414045 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9131d-ffb0-4017-bbca-de6f6074283a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:14.414722 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9131d-ffb0-4017-bbca-de6f6074283a None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:14.415006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9131d-ffb0-4017-bbca-de6f6074283a None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:14.415343 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9131d-ffb0-4017-bbca-de6f6074283a None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:14.416057 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9131d-ffb0-4017-bbca-de6f6074283a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:14.416359 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9131d-ffb0-4017-bbca-de6f6074283a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:14.417810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9131d-ffb0-4017-bbca-de6f6074283a None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:14.428746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 632/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:14.434651 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34988dc6-8dec-4211-b7b4-0384e20c931c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:14.518077 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:14.539225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34988dc6-8dec-4211-b7b4-0384e20c931c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:14.572291 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f73c59-db7b-412d-b39a-7a75ba89e2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:14.573721 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f73c59-db7b-412d-b39a-7a75ba89e2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:14.574003 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f73c59-db7b-412d-b39a-7a75ba89e2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:14.574282 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f73c59-db7b-412d-b39a-7a75ba89e2f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:14.581869 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f73c59-db7b-412d-b39a-7a75ba89e2f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:14.583387 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f73c59-db7b-412d-b39a-7a75ba89e2f2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:14.597109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-f8f73c59-db7b-412d-b39a-7a75ba89e2f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:29:14.597109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:14.597109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:14.597109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:14.597109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:14.597109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:14.597109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:14.597109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:14.597109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:14.597109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:14.597109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:14.597109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:14.597109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:29:14.597109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:29:14.597109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:29:14.598122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:29:14.598122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:14.598122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:14.598122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:14.598122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:14.598122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:14.598122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:14.598122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:14.598122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:14.598122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:14.598122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:14.598122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:14.598122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:14.598122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:14.598122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:14.598122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:14.598122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:14.599032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:29:14.599032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:29:14.599032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:29:14.599032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:29:14.599032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:29:14.599032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:29:14.599032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:29:14.599032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:14.599882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 612/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:29:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:14.604173 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9131d-ffb0-4017-bbca-de6f6074283a None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:14.703662 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9131d-ffb0-4017-bbca-de6f6074283a None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:14.736195 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2cdd4-7040-4c24-a98f-516cd0e798ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:14.736887 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2cdd4-7040-4c24-a98f-516cd0e798ac None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:14.737123 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2cdd4-7040-4c24-a98f-516cd0e798ac None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:14.737398 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2cdd4-7040-4c24-a98f-516cd0e798ac None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:14.738169 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2cdd4-7040-4c24-a98f-516cd0e798ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:14.738478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2cdd4-7040-4c24-a98f-516cd0e798ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:14.739928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2cdd4-7040-4c24-a98f-516cd0e798ac None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:14.752190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 633/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:29:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:14.758320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f73c59-db7b-412d-b39a-7a75ba89e2f2 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:14.857790 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f73c59-db7b-412d-b39a-7a75ba89e2f2 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:14.891855 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e8913-fe1e-4b7b-bb96-a946c483c19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:14.892491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e8913-fe1e-4b7b-bb96-a946c483c19d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:14.892775 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e8913-fe1e-4b7b-bb96-a946c483c19d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:14.893045 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e8913-fe1e-4b7b-bb96-a946c483c19d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:14.893867 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e8913-fe1e-4b7b-bb96-a946c483c19d None admin] RBAC: Adding query filter params (role.id=13708a4819a94b94b47ba5994aaa9310, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=ac04b32777ca4f929e9e59deee6e0260) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:14.894177 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e8913-fe1e-4b7b-bb96-a946c483c19d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:14.895675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e8913-fe1e-4b7b-bb96-a946c483c19d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:14.903069 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 613/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:29:14 2022] GET /identity/v3/role_assignments?user.id=ac04b32777ca4f929e9e59deee6e0260&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=13708a4819a94b94b47ba5994aaa9310 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:16.969686 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912c1e7-93d6-4ddd-af2d-03c09fb34099 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:16.970215 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912c1e7-93d6-4ddd-af2d-03c09fb34099 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:16.970215 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912c1e7-93d6-4ddd-af2d-03c09fb34099 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:16.971765 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 634/1247] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:29:16.976974 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5234e42-43d6-4e94-8af4-2dfaa57958b9 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:16.977398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5234e42-43d6-4e94-8af4-2dfaa57958b9 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:16.977528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5234e42-43d6-4e94-8af4-2dfaa57958b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:16.983591 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:16.995529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c5234e42-43d6-4e94-8af4-2dfaa57958b9 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:29:16.996725 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c5234e42-43d6-4e94-8af4-2dfaa57958b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:29:17.061710 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 614/1248] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:29:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:29:17.085518 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912c1e7-93d6-4ddd-af2d-03c09fb34099 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:17.103790 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:17.197074 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912c1e7-93d6-4ddd-af2d-03c09fb34099 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:17.232158 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302c7a6-0d0a-418f-a660-62f7e6b41cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:17.232877 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302c7a6-0d0a-418f-a660-62f7e6b41cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:17.233158 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302c7a6-0d0a-418f-a660-62f7e6b41cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:17.233428 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302c7a6-0d0a-418f-a660-62f7e6b41cb3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:17.242978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302c7a6-0d0a-418f-a660-62f7e6b41cb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:29:17.251113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-c302c7a6-0d0a-418f-a660-62f7e6b41cb3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:29:17.251113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:17.251113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:17.251113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:17.251113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:17.251113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:17.251113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:17.251113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:17.251113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:17.251113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:17.251113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:17.251113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:17.251113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:29:17.251113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:29:17.251113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:29:17.251113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:29:17.255919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:29:17.255919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:29:17.255919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:29:17.255919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:29:17.255919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:17.255919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:17.255919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:17.255919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:17.255919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:17.255919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:17.255919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:17.255919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:17.255919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:17.255919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:17.255919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:17.255919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:17.256998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:17.256998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:17.256998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:17.256998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:17.256998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:29:17.256998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:29:17.256998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:29:17.256998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:29:17.256998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:29:17.256998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:29:17.256998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:29:17.256998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:29:17.256998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 635/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:29:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:17.259317 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5234e42-43d6-4e94-8af4-2dfaa57958b9 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:17.357629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5234e42-43d6-4e94-8af4-2dfaa57958b9 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:17.386490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:17.391284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f125b4-5e74-4731-a71f-7e856a4a704d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:17.391942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f125b4-5e74-4731-a71f-7e856a4a704d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:17.392225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f125b4-5e74-4731-a71f-7e856a4a704d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:17.392499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f125b4-5e74-4731-a71f-7e856a4a704d None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:17.393239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f125b4-5e74-4731-a71f-7e856a4a704d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:17.393551 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f125b4-5e74-4731-a71f-7e856a4a704d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:17.394966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f125b4-5e74-4731-a71f-7e856a4a704d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:17.403493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 615/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:29:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:17.409365 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302c7a6-0d0a-418f-a660-62f7e6b41cb3 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:17.490844 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:17.513479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302c7a6-0d0a-418f-a660-62f7e6b41cb3 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:17.545781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c60b49-ecc5-4ca9-928a-d24604d82478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:17.546578 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c60b49-ecc5-4ca9-928a-d24604d82478 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:17.547076 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c60b49-ecc5-4ca9-928a-d24604d82478 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:17.547409 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c60b49-ecc5-4ca9-928a-d24604d82478 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:17.558728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c60b49-ecc5-4ca9-928a-d24604d82478 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:17.560188 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c60b49-ecc5-4ca9-928a-d24604d82478 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:17.577934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-64c60b49-ecc5-4ca9-928a-d24604d82478 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:17.577934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:17.577934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:17.577934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:17.577934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:17.577934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:17.577934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:17.577934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:17.577934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:17.577934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:17.577934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:17.577934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:17.577934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:29:17.577934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:29:17.577934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:29:17.579059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:29:17.579059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:17.579059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:17.579059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:17.579059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:17.579059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:17.579059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:17.579059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:17.579059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:17.579059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:17.579059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:17.579059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:17.579059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:17.579059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:17.579059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:17.579059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:17.579059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:17.580176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:29:17.580176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:29:17.580176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:17.580176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:29:17.580176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 636/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:17.584957 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f125b4-5e74-4731-a71f-7e856a4a704d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:17.683943 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f125b4-5e74-4731-a71f-7e856a4a704d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:17.716905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7468c-eae0-492f-a7ca-e1c881ba330f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:17.717563 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7468c-eae0-492f-a7ca-e1c881ba330f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:17.717845 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7468c-eae0-492f-a7ca-e1c881ba330f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:17.718142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7468c-eae0-492f-a7ca-e1c881ba330f None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:17.718896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7468c-eae0-492f-a7ca-e1c881ba330f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:17.719203 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7468c-eae0-492f-a7ca-e1c881ba330f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:17.720563 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7468c-eae0-492f-a7ca-e1c881ba330f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:17.732180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 616/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:17.737777 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c60b49-ecc5-4ca9-928a-d24604d82478 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:17.836982 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c60b49-ecc5-4ca9-928a-d24604d82478 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:17.861890 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:17.869446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a91bf5-da2b-4263-97fc-6200313b8ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:17.870840 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a91bf5-da2b-4263-97fc-6200313b8ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:17.871097 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a91bf5-da2b-4263-97fc-6200313b8ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:17.871366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a91bf5-da2b-4263-97fc-6200313b8ab1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:17.878420 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a91bf5-da2b-4263-97fc-6200313b8ab1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:17.879896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a91bf5-da2b-4263-97fc-6200313b8ab1 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:17.894144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-c9a91bf5-da2b-4263-97fc-6200313b8ab1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:29:17.894144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:17.894144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:17.894144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:17.894144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:17.894144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:17.894144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:17.894144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:17.894144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:17.894144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:17.894144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:17.894144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:17.894144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:29:17.894144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:29:17.894144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:29:17.894144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:29:17.895422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:17.895422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:17.895422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:29:17.895422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:29:17.895422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:29:17.895422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:29:17.895422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:17.895422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:17.895422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:17.895422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:17.895422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:17.895422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:17.895422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:17.895422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:17.895422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:17.895422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:17.895422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:17.895422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:17.896391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:17.896391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:17.896391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:29:17.896391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:29:17.896391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:29:17.896391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:29:17.896391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:29:17.896391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:29:17.896391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:29:17.896391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:29:17.896391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 637/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:29:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:17.901170 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7468c-eae0-492f-a7ca-e1c881ba330f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:17.956617 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:18.000579 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7468c-eae0-492f-a7ca-e1c881ba330f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:18.033675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a151ea3-6ec9-46dd-bfea-f92a61c24e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:18.034330 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a151ea3-6ec9-46dd-bfea-f92a61c24e80 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:18.034648 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a151ea3-6ec9-46dd-bfea-f92a61c24e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:18.034924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a151ea3-6ec9-46dd-bfea-f92a61c24e80 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:18.035744 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a151ea3-6ec9-46dd-bfea-f92a61c24e80 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:18.036050 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a151ea3-6ec9-46dd-bfea-f92a61c24e80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:18.037531 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a151ea3-6ec9-46dd-bfea-f92a61c24e80 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:18.055677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 617/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Jun 23 11:29:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:18.062142 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a91bf5-da2b-4263-97fc-6200313b8ab1 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:18.159349 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a91bf5-da2b-4263-97fc-6200313b8ab1 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:18.192932 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2a2e0-f855-4974-9af1-337ac9aaa069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:18.193568 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2a2e0-f855-4974-9af1-337ac9aaa069 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:18.193851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2a2e0-f855-4974-9af1-337ac9aaa069 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:18.194128 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2a2e0-f855-4974-9af1-337ac9aaa069 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:18.247493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:18.259372 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2a2e0-f855-4974-9af1-337ac9aaa069 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:18.261016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2a2e0-f855-4974-9af1-337ac9aaa069 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:18.356138 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 638/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:29:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:29:18.362065 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a151ea3-6ec9-46dd-bfea-f92a61c24e80 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:18.460514 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a151ea3-6ec9-46dd-bfea-f92a61c24e80 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:18.474228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:18.495510 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b27a80-fe84-4cdc-8569-8f238a9f5f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:18.496304 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b27a80-fe84-4cdc-8569-8f238a9f5f1a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:18.496587 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b27a80-fe84-4cdc-8569-8f238a9f5f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:18.496859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b27a80-fe84-4cdc-8569-8f238a9f5f1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:18.506839 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b27a80-fe84-4cdc-8569-8f238a9f5f1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:18.508284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b27a80-fe84-4cdc-8569-8f238a9f5f1a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:18.525406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-68b27a80-fe84-4cdc-8569-8f238a9f5f1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:18.525406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:18.525406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:18.525406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:18.525406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:18.525406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:18.525406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:18.525406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:18.525406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:18.525406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:18.525406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:18.525406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:18.525406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:29:18.525406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:29:18.525406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:29:18.527398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:29:18.527398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:18.527398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:18.527398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:18.527398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:18.527398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:18.527398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:18.527398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:18.527398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:18.527398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:18.527398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:18.527398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:18.527398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:18.527398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:18.527398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:18.527398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:18.527398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:18.528482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:29:18.528482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:29:18.528482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:18.528482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:18.528482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 618/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:29:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:18.534817 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2a2e0-f855-4974-9af1-337ac9aaa069 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:18.620206 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:18.631659 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2a2e0-f855-4974-9af1-337ac9aaa069 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:18.664260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033ea00-fc21-4c68-bf01-857be9f1e3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:18.664904 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033ea00-fc21-4c68-bf01-857be9f1e3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:18.665183 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033ea00-fc21-4c68-bf01-857be9f1e3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:18.665455 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033ea00-fc21-4c68-bf01-857be9f1e3d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:18.666164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5033ea00-fc21-4c68-bf01-857be9f1e3d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:18.666605 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5033ea00-fc21-4c68-bf01-857be9f1e3d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:18.668601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5033ea00-fc21-4c68-bf01-857be9f1e3d5 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:18.679537 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 639/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:29:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:18.685425 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b27a80-fe84-4cdc-8569-8f238a9f5f1a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:18.783210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b27a80-fe84-4cdc-8569-8f238a9f5f1a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:18.816239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10402af5-0e2d-409d-9a66-0aee9b7a4424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:18.817015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10402af5-0e2d-409d-9a66-0aee9b7a4424 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:18.817293 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10402af5-0e2d-409d-9a66-0aee9b7a4424 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:18.817566 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10402af5-0e2d-409d-9a66-0aee9b7a4424 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:18.824636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10402af5-0e2d-409d-9a66-0aee9b7a4424 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:18.826100 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10402af5-0e2d-409d-9a66-0aee9b7a4424 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:18.839936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-10402af5-0e2d-409d-9a66-0aee9b7a4424 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:29:18.839936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:18.839936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:18.839936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:18.839936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:18.839936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:18.839936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:18.839936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:18.839936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:18.839936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:18.839936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:18.839936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:18.839936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:29:18.839936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:29:18.839936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:29:18.840981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:29:18.840981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:18.840981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:18.840981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:18.840981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:18.840981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:18.840981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:18.840981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:18.840981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:18.840981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:18.840981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:18.840981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:18.840981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:18.840981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:18.840981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:18.840981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:18.840981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:18.842259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:29:18.842259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:29:18.842259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:29:18.842259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:29:18.842259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:29:18.842259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:29:18.842259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:29:18.842259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:18.842259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 619/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:29:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:18.846974 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033ea00-fc21-4c68-bf01-857be9f1e3d5 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:18.947782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033ea00-fc21-4c68-bf01-857be9f1e3d5 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:18.982274 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776802e9-bf62-4d34-aab7-3bd3015d04ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:18.982947 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776802e9-bf62-4d34-aab7-3bd3015d04ad None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:18.983231 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776802e9-bf62-4d34-aab7-3bd3015d04ad None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:18.983503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776802e9-bf62-4d34-aab7-3bd3015d04ad None admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:18.984274 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776802e9-bf62-4d34-aab7-3bd3015d04ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:18.984731 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776802e9-bf62-4d34-aab7-3bd3015d04ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:18.986159 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776802e9-bf62-4d34-aab7-3bd3015d04ad None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:18.998456 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 640/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:29:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:19.004544 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10402af5-0e2d-409d-9a66-0aee9b7a4424 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:19.032147 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:19.106276 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10402af5-0e2d-409d-9a66-0aee9b7a4424 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:19.141834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d237a5-e2f8-40d3-9f2e-467bb0560fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:19.142481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d237a5-e2f8-40d3-9f2e-467bb0560fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:19.142789 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d237a5-e2f8-40d3-9f2e-467bb0560fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:19.143063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d237a5-e2f8-40d3-9f2e-467bb0560fb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:19.143885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d237a5-e2f8-40d3-9f2e-467bb0560fb1 None admin] RBAC: Adding query filter params (role.id=c9e006de0d6c4812ab29d778e9c8a830, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=ac04b32777ca4f929e9e59deee6e0260) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:19.144191 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d237a5-e2f8-40d3-9f2e-467bb0560fb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:19.145613 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d237a5-e2f8-40d3-9f2e-467bb0560fb1 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:19.151115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 620/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:29:19 2022] GET /identity/v3/role_assignments?user.id=ac04b32777ca4f929e9e59deee6e0260&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=c9e006de0d6c4812ab29d778e9c8a830 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:21.152102 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147aab0-02c2-45ac-b1cf-43c0b9c45c6f None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:21.152636 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147aab0-02c2-45ac-b1cf-43c0b9c45c6f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:21.152756 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147aab0-02c2-45ac-b1cf-43c0b9c45c6f None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:21.154026 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 641/1261] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:29:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:29:21.159231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10926d-3682-4af5-8d53-1638f43364b8 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:21.159635 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10926d-3682-4af5-8d53-1638f43364b8 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:21.159785 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10926d-3682-4af5-8d53-1638f43364b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:21.177179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-bd10926d-3682-4af5-8d53-1638f43364b8 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:29:21.181400 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bd10926d-3682-4af5-8d53-1638f43364b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:29:21.244172 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 621/1262] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:29:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:29:21.267892 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147aab0-02c2-45ac-b1cf-43c0b9c45c6f None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:21.279185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:21.367796 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147aab0-02c2-45ac-b1cf-43c0b9c45c6f None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:21.400106 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21822eab-2436-4b41-a7bb-5b31437b087f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:21.400902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21822eab-2436-4b41-a7bb-5b31437b087f None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:21.401172 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21822eab-2436-4b41-a7bb-5b31437b087f None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:21.402407 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21822eab-2436-4b41-a7bb-5b31437b087f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:21.412122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21822eab-2436-4b41-a7bb-5b31437b087f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:29:21.420779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-21822eab-2436-4b41-a7bb-5b31437b087f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:29:21.420779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:21.420779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:21.420779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:21.420779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:21.420779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:21.420779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:21.420779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:21.420779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:21.420779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:21.420779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:21.420779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:21.420779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:29:21.420779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:29:21.420779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:29:21.420779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:29:21.421891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:29:21.421891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:29:21.421891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:29:21.421891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:29:21.421891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:21.421891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:21.421891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:21.421891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:21.421891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:21.421891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:21.421891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:21.421891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:21.421891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:21.421891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:21.421891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:21.421891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:21.422924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:21.422924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:21.422924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:21.422924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:21.422924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:29:21.422924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:29:21.422924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:29:21.422924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:29:21.422924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:29:21.422924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:29:21.422924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:29:21.422924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:29:21.422924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 642/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:29:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:21.428215 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10926d-3682-4af5-8d53-1638f43364b8 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:21.446322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:21.528850 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10926d-3682-4af5-8d53-1638f43364b8 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:21.562593 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a31a7-e77f-4d32-937d-39e7e502d1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:21.563259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a31a7-e77f-4d32-937d-39e7e502d1dc None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:21.563541 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a31a7-e77f-4d32-937d-39e7e502d1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:21.563815 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a31a7-e77f-4d32-937d-39e7e502d1dc None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:21.564554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a31a7-e77f-4d32-937d-39e7e502d1dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:21.565115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a31a7-e77f-4d32-937d-39e7e502d1dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:21.566480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a31a7-e77f-4d32-937d-39e7e502d1dc None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:21.574868 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 622/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:29:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:21.580513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21822eab-2436-4b41-a7bb-5b31437b087f None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:21.646523 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:21.679434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21822eab-2436-4b41-a7bb-5b31437b087f None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:21.711681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb270d-ee8f-4a94-b700-b2132a161a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:21.712460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb270d-ee8f-4a94-b700-b2132a161a06 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:21.712745 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb270d-ee8f-4a94-b700-b2132a161a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:21.713021 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb270d-ee8f-4a94-b700-b2132a161a06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:21.723624 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb270d-ee8f-4a94-b700-b2132a161a06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:21.725093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb270d-ee8f-4a94-b700-b2132a161a06 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:21.741920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-5deb270d-ee8f-4a94-b700-b2132a161a06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:21.741920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:21.741920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:21.741920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:21.741920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:21.741920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:21.741920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:21.741920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:21.741920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:21.741920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:21.741920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:21.741920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:21.741920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:29:21.741920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:29:21.741920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:29:21.744647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:29:21.744647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:21.744647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:21.744647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:21.744647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:21.744647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:21.744647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:21.744647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:21.744647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:21.744647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:21.744647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:21.744647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:21.744647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:21.744647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:21.744647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:21.744647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:21.744647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:21.746462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:29:21.746462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:29:21.746462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:21.746462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:29:21.746462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 643/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:29:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:21.749342 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a31a7-e77f-4d32-937d-39e7e502d1dc None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:21.828940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:21.847420 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a31a7-e77f-4d32-937d-39e7e502d1dc None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:21.879413 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603effa-a566-4a87-bc88-5840fbcf4ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:21.880076 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603effa-a566-4a87-bc88-5840fbcf4ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:21.880375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603effa-a566-4a87-bc88-5840fbcf4ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:21.880632 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603effa-a566-4a87-bc88-5840fbcf4ba7 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:21.881343 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603effa-a566-4a87-bc88-5840fbcf4ba7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:21.881682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603effa-a566-4a87-bc88-5840fbcf4ba7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:21.883074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603effa-a566-4a87-bc88-5840fbcf4ba7 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:21.894466 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 623/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:21.900110 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb270d-ee8f-4a94-b700-b2132a161a06 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:21.999030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb270d-ee8f-4a94-b700-b2132a161a06 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:22.018653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:22.032822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f4830-6c71-4600-92ea-173c62c433de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:22.033610 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f4830-6c71-4600-92ea-173c62c433de None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:22.033890 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f4830-6c71-4600-92ea-173c62c433de None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:22.034162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f4830-6c71-4600-92ea-173c62c433de None admin] PATH_INFO: `/v3/users/placement` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:22.042494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f4830-6c71-4600-92ea-173c62c433de None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:22.044007 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f4830-6c71-4600-92ea-173c62c433de None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:22.058328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-597f4830-6c71-4600-92ea-173c62c433de None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:29:22.058328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:22.058328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:22.058328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:22.058328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:22.058328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:22.058328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:22.058328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:22.058328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:22.058328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:22.058328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:22.058328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:22.058328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:29:22.058328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:29:22.058328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:29:22.058328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:29:22.072028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:22.072028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:22.072028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:29:22.072028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:29:22.072028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:29:22.072028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:29:22.072028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:22.072028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:22.072028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:22.072028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:22.072028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:22.072028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:22.072028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:22.072028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:22.072028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:22.072028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:22.072028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:22.072028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:22.073851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:22.073851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:22.073851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:29:22.073851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:29:22.073851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:29:22.073851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:29:22.073851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:29:22.073851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:29:22.073851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:29:22.073851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:29:22.073851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 644/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:29:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:22.074820 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603effa-a566-4a87-bc88-5840fbcf4ba7 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:22.163507 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603effa-a566-4a87-bc88-5840fbcf4ba7 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:22.196170 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0c761-dc6e-4ef9-b0d2-321da07f374f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:22.196830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0c761-dc6e-4ef9-b0d2-321da07f374f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:22.197110 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0c761-dc6e-4ef9-b0d2-321da07f374f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:22.197671 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0c761-dc6e-4ef9-b0d2-321da07f374f None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:22.198265 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0c761-dc6e-4ef9-b0d2-321da07f374f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:22.199152 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0c761-dc6e-4ef9-b0d2-321da07f374f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:22.200574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0c761-dc6e-4ef9-b0d2-321da07f374f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:22.209685 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:22.218030 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 624/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Jun 23 11:29:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:22.223617 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f4830-6c71-4600-92ea-173c62c433de None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:22.319526 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f4830-6c71-4600-92ea-173c62c433de None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:22.352348 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fc4c3-d182-4a76-b06e-7d0fa8b50281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:22.353127 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fc4c3-d182-4a76-b06e-7d0fa8b50281 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:22.353411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fc4c3-d182-4a76-b06e-7d0fa8b50281 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:22.353686 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fc4c3-d182-4a76-b06e-7d0fa8b50281 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:22.364380 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fc4c3-d182-4a76-b06e-7d0fa8b50281 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:22.365857 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fc4c3-d182-4a76-b06e-7d0fa8b50281 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:22.383247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-643fc4c3-d182-4a76-b06e-7d0fa8b50281 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:22.383247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:22.383247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:22.383247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:22.383247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:22.383247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:22.383247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:22.383247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:22.383247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:22.383247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:22.383247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:22.383247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:22.383247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:29:22.383247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:29:22.383247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:29:22.384387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:29:22.384387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:22.384387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:22.384387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:22.384387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:22.384387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:22.384387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:22.384387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:22.384387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:22.384387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:22.384387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:22.384387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:22.384387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:22.384387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:22.384387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:22.384387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:22.384387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:22.385517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:29:22.385517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:29:22.385517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:22.385517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:29:22.385517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 645/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:29:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:22.390892 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0c761-dc6e-4ef9-b0d2-321da07f374f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:22.494780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0c761-dc6e-4ef9-b0d2-321da07f374f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:22.530370 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa80ba-d33f-4e8b-8103-98e644f54ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:22.531064 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa80ba-d33f-4e8b-8103-98e644f54ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:22.531354 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa80ba-d33f-4e8b-8103-98e644f54ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:22.531633 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa80ba-d33f-4e8b-8103-98e644f54ea9 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:22.532465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa80ba-d33f-4e8b-8103-98e644f54ea9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:22.532776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa80ba-d33f-4e8b-8103-98e644f54ea9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:22.534153 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa80ba-d33f-4e8b-8103-98e644f54ea9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:22.546978 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 625/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:22.552813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fc4c3-d182-4a76-b06e-7d0fa8b50281 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:22.588327 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:22.666896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fc4c3-d182-4a76-b06e-7d0fa8b50281 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:22.709846 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d8669-5c4b-4d6b-bc12-c5f9fecb3683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:22.710662 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d8669-5c4b-4d6b-bc12-c5f9fecb3683 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:22.710945 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d8669-5c4b-4d6b-bc12-c5f9fecb3683 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:22.711219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d8669-5c4b-4d6b-bc12-c5f9fecb3683 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:22.719047 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d8669-5c4b-4d6b-bc12-c5f9fecb3683 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:22.720744 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d8669-5c4b-4d6b-bc12-c5f9fecb3683 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:22.735938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-9e6d8669-5c4b-4d6b-bc12-c5f9fecb3683 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:29:22.735938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:22.735938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:22.735938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:22.735938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:22.735938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:22.735938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:22.735938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:22.735938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:22.735938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:22.735938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:22.735938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:22.735938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:29:22.735938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:29:22.735938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:29:22.737013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:29:22.737013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:22.737013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:22.737013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:22.737013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:22.737013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:22.737013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:22.737013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:22.737013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:22.737013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:22.737013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:22.737013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:22.737013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:22.737013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:22.737013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:22.737013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:22.737013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:22.743369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:29:22.743369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:29:22.743369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:29:22.743369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:29:22.743369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:29:22.743369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:29:22.743369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:29:22.743369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:29:22.743369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 646/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:29:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:22.744057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa80ba-d33f-4e8b-8103-98e644f54ea9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:22.821309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:22.866098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa80ba-d33f-4e8b-8103-98e644f54ea9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:22.901807 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8211e032-3e91-47b4-b373-4014f947f7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:22.902770 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211e032-3e91-47b4-b373-4014f947f7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:22.903057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211e032-3e91-47b4-b373-4014f947f7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:22.903333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211e032-3e91-47b4-b373-4014f947f7d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:22.904123 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8211e032-3e91-47b4-b373-4014f947f7d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:22.904434 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8211e032-3e91-47b4-b373-4014f947f7d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:22.905868 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8211e032-3e91-47b4-b373-4014f947f7d9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:22.919709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 626/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:29:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:22.925800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d8669-5c4b-4d6b-bc12-c5f9fecb3683 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:23.031174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:23.039709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d8669-5c4b-4d6b-bc12-c5f9fecb3683 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:23.076887 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7abfd1-fd02-4fdb-a54c-cfab0d9cda49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:23.077782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7abfd1-fd02-4fdb-a54c-cfab0d9cda49 None admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:23.078062 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7abfd1-fd02-4fdb-a54c-cfab0d9cda49 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:23.078335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7abfd1-fd02-4fdb-a54c-cfab0d9cda49 None admin] PATH_INFO: `/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/ac04b32777ca4f929e9e59deee6e0260/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:23.080301 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7abfd1-fd02-4fdb-a54c-cfab0d9cda49 None admin] RBAC: Authorizing `identity:create_grant(project_id=886c2939a14b48febc2e2fe639c8f1e4, user_id=ac04b32777ca4f929e9e59deee6e0260, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:23.081730 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7abfd1-fd02-4fdb-a54c-cfab0d9cda49 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:23.233839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 647/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 23 11:29:22 2022] PUT /identity/v3/projects/886c2939a14b48febc2e2fe639c8f1e4/users/ac04b32777ca4f929e9e59deee6e0260/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:29:25.274771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdbffa-8c00-4510-8c8b-7fc30cf06698 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:25.275299 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdbffa-8c00-4510-8c8b-7fc30cf06698 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:25.275299 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdbffa-8c00-4510-8c8b-7fc30cf06698 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:25.276891 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 627/1274] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:29:25.281806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018d97d-c088-4752-bee5-cb9c1fcfc522 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:25.282226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018d97d-c088-4752-bee5-cb9c1fcfc522 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:25.282382 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018d97d-c088-4752-bee5-cb9c1fcfc522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:25.299418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c018d97d-c088-4752-bee5-cb9c1fcfc522 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:29:25.300791 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c018d97d-c088-4752-bee5-cb9c1fcfc522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:29:25.372239 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 648/1275] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:29:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:29:25.396261 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdbffa-8c00-4510-8c8b-7fc30cf06698 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:25.502282 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdbffa-8c00-4510-8c8b-7fc30cf06698 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:25.519892 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:25.536601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436081f9-0440-4f22-84b2-0f209100ee7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:25.537327 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436081f9-0440-4f22-84b2-0f209100ee7d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:25.537611 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436081f9-0440-4f22-84b2-0f209100ee7d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:25.537886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436081f9-0440-4f22-84b2-0f209100ee7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:25.548414 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436081f9-0440-4f22-84b2-0f209100ee7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:29:25.556288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-436081f9-0440-4f22-84b2-0f209100ee7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:29:25.556288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:25.556288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:25.556288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:25.556288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:25.556288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:25.556288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:25.556288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:25.556288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:25.556288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:25.556288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:25.556288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:25.556288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:29:25.556288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:29:25.556288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:29:25.556288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:29:25.559490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:29:25.559490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:29:25.559490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:29:25.559490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:29:25.559490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:25.559490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:25.559490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:25.559490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:25.559490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:25.559490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:25.559490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:25.559490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:25.559490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:25.559490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:25.559490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:25.559490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:25.560392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:25.560392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:25.560392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:25.560392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:25.560392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:29:25.560392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:29:25.560392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:29:25.560392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:29:25.560392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:29:25.560392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:29:25.560392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:29:25.560392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:25.561801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 628/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 23 11:29:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:25.564317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018d97d-c088-4752-bee5-cb9c1fcfc522 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:25.662038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:25.679968 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018d97d-c088-4752-bee5-cb9c1fcfc522 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:25.713488 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75b83e-172c-4b95-ba1d-27378511691b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:25.714659 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75b83e-172c-4b95-ba1d-27378511691b None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:25.714946 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75b83e-172c-4b95-ba1d-27378511691b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:25.715222 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75b83e-172c-4b95-ba1d-27378511691b None admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:25.715955 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75b83e-172c-4b95-ba1d-27378511691b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:25.716289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75b83e-172c-4b95-ba1d-27378511691b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:25.717741 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75b83e-172c-4b95-ba1d-27378511691b None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:25.725426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 649/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 23 11:29:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:25.731665 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436081f9-0440-4f22-84b2-0f209100ee7d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:25.829702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436081f9-0440-4f22-84b2-0f209100ee7d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:25.865379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be2b0e-f1bc-4ba5-a093-64cc558b6c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:25.866167 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be2b0e-f1bc-4ba5-a093-64cc558b6c44 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:25.866450 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be2b0e-f1bc-4ba5-a093-64cc558b6c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:25.866750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be2b0e-f1bc-4ba5-a093-64cc558b6c44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:25.877427 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be2b0e-f1bc-4ba5-a093-64cc558b6c44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:25.878976 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be2b0e-f1bc-4ba5-a093-64cc558b6c44 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:25.895995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-f2be2b0e-f1bc-4ba5-a093-64cc558b6c44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:25.895995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:25.895995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:25.895995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:25.895995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:25.895995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:25.895995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:25.895995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:25.895995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:25.895995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:25.895995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:25.895995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:25.895995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:29:25.895995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:29:25.895995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:29:25.897101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:29:25.897101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:25.897101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:25.897101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:25.897101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:25.897101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:25.897101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:25.897101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:25.897101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:25.897101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:25.897101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:25.897101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:25.897101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:25.897101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:25.897101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:25.897101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:25.897101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:25.898222 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:29:25.898222 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:29:25.898222 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:25.898222 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:25.898222 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 629/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:29:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:25.905374 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75b83e-172c-4b95-ba1d-27378511691b None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:26.005266 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75b83e-172c-4b95-ba1d-27378511691b None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:26.038929 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81154214-5ace-4e08-a988-1ceabab46eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:26.039582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81154214-5ace-4e08-a988-1ceabab46eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:26.039876 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81154214-5ace-4e08-a988-1ceabab46eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:26.040148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81154214-5ace-4e08-a988-1ceabab46eb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:26.040861 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81154214-5ace-4e08-a988-1ceabab46eb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:26.041165 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81154214-5ace-4e08-a988-1ceabab46eb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:26.043752 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81154214-5ace-4e08-a988-1ceabab46eb3 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:26.056203 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 650/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:26.061885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be2b0e-f1bc-4ba5-a093-64cc558b6c44 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:26.094320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:26.162811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be2b0e-f1bc-4ba5-a093-64cc558b6c44 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:26.196406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7551353-74db-433e-913b-c00955b03da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:26.197191 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7551353-74db-433e-913b-c00955b03da0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:26.197474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7551353-74db-433e-913b-c00955b03da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:26.197743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7551353-74db-433e-913b-c00955b03da0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:26.205366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7551353-74db-433e-913b-c00955b03da0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:26.206831 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7551353-74db-433e-913b-c00955b03da0 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:26.222059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-d7551353-74db-433e-913b-c00955b03da0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:29:26.222059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:26.222059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:26.222059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:26.222059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:26.222059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:26.222059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:26.222059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:26.222059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:26.222059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:26.222059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:26.222059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:26.222059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:29:26.222059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:29:26.222059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:29:26.222059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:29:26.226675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:26.226675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:26.226675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:29:26.226675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:29:26.226675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:29:26.226675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:29:26.226675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:26.226675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:26.226675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:26.226675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:26.226675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:26.226675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:26.226675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:26.226675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:26.226675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:26.226675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:26.226675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:26.226675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:26.227847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:26.227847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:26.227847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:29:26.227847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:29:26.227847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:29:26.227847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:29:26.227847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:29:26.227847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:29:26.227847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:29:26.227847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:26.227847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 630/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:29:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:26.229353 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81154214-5ace-4e08-a988-1ceabab46eb3 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:26.233587 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:26.327515 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81154214-5ace-4e08-a988-1ceabab46eb3 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:26.360343 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c72344-910b-46cf-9503-c89c9c41beaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:26.360936 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c72344-910b-46cf-9503-c89c9c41beaf None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:26.361216 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c72344-910b-46cf-9503-c89c9c41beaf None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:26.361490 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c72344-910b-46cf-9503-c89c9c41beaf None admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:26.362374 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c72344-910b-46cf-9503-c89c9c41beaf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:26.362712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c72344-910b-46cf-9503-c89c9c41beaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:26.364129 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c72344-910b-46cf-9503-c89c9c41beaf None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:26.380535 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 651/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Jun 23 11:29:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:29:26.387373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7551353-74db-433e-913b-c00955b03da0 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:26.479219 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:26.489281 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7551353-74db-433e-913b-c00955b03da0 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:26.522951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3cd591-bb16-4952-a093-b9bc490b55b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:26.523606 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3cd591-bb16-4952-a093-b9bc490b55b3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:26.523893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3cd591-bb16-4952-a093-b9bc490b55b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:26.524851 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3cd591-bb16-4952-a093-b9bc490b55b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:26.592733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3cd591-bb16-4952-a093-b9bc490b55b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:26.594208 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3cd591-bb16-4952-a093-b9bc490b55b3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:26.680695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:26.691666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 631/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 23 11:29:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:29:26.697492 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c72344-910b-46cf-9503-c89c9c41beaf None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:26.758595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:26.796901 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c72344-910b-46cf-9503-c89c9c41beaf None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:26.830129 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5221439-ebae-48ff-8d52-06cfa0ae4d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:26.830941 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5221439-ebae-48ff-8d52-06cfa0ae4d31 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:26.831227 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5221439-ebae-48ff-8d52-06cfa0ae4d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:26.831503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5221439-ebae-48ff-8d52-06cfa0ae4d31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:26.841618 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5221439-ebae-48ff-8d52-06cfa0ae4d31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:26.843113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5221439-ebae-48ff-8d52-06cfa0ae4d31 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:26.860335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-b5221439-ebae-48ff-8d52-06cfa0ae4d31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:26.860335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:26.860335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:26.860335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:26.860335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:26.860335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:26.860335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:26.860335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:26.860335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:26.860335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:26.860335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:26.860335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:26.860335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:29:26.860335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:29:26.860335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:29:26.861473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:29:26.861473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:26.861473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:26.861473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:26.861473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:26.861473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:26.861473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:26.861473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:26.861473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:26.861473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:26.861473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:26.861473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:26.861473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:26.861473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:26.861473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:26.861473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:26.861473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:26.862558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:29:26.862558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:29:26.862558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:29:26.862558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:29:26.862558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 652/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:29:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:26.867249 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3cd591-bb16-4952-a093-b9bc490b55b3 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:26.966955 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3cd591-bb16-4952-a093-b9bc490b55b3 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:27.002867 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dacd9-04bc-4630-99b0-9923d918f549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:27.003527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dacd9-04bc-4630-99b0-9923d918f549 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:27.003814 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dacd9-04bc-4630-99b0-9923d918f549 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:27.004086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dacd9-04bc-4630-99b0-9923d918f549 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:27.004804 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545dacd9-04bc-4630-99b0-9923d918f549 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:27.005112 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545dacd9-04bc-4630-99b0-9923d918f549 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:27.006477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545dacd9-04bc-4630-99b0-9923d918f549 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:27.017491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 632/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 23 11:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:27.023238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5221439-ebae-48ff-8d52-06cfa0ae4d31 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:27.123475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5221439-ebae-48ff-8d52-06cfa0ae4d31 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:27.136444 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:27.155411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07191cd-75b7-4d29-b5d1-174f1545ffa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:27.156182 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07191cd-75b7-4d29-b5d1-174f1545ffa0 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:27.156463 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07191cd-75b7-4d29-b5d1-174f1545ffa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:27.156737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07191cd-75b7-4d29-b5d1-174f1545ffa0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:27.163135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07191cd-75b7-4d29-b5d1-174f1545ffa0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:27.164573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07191cd-75b7-4d29-b5d1-174f1545ffa0 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:27.178764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application [None req-a07191cd-75b7-4d29-b5d1-174f1545ffa0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:29:27.178764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:27.178764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:27.178764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:27.178764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:27.178764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:27.178764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:27.178764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:27.178764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:27.178764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:27.178764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:27.178764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:27.178764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:29:27.178764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:29:27.178764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:29:27.179815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:29:27.179815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:27.179815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:27.179815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:27.179815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:27.179815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:27.179815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:27.179815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:27.179815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:27.179815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:27.179815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:27.179815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:27.179815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:27.179815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:27.179815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:27.179815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:27.179815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:27.181067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:29:27.181067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:29:27.181067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:29:27.181067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:29:27.181067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:29:27.181067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:29:27.181067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:29:27.181067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: ERROR keystone.server.flask.application Jun 23 11:29:27.181067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 653/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 23 11:29:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:27.186336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dacd9-04bc-4630-99b0-9923d918f549 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:27.236135 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:27.285590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dacd9-04bc-4630-99b0-9923d918f549 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:27.319871 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf026ca0-2ea7-4e98-b11b-712e6bd319e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:27.320530 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf026ca0-2ea7-4e98-b11b-712e6bd319e9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:27.320812 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf026ca0-2ea7-4e98-b11b-712e6bd319e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:27.321083 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf026ca0-2ea7-4e98-b11b-712e6bd319e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:27.321957 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf026ca0-2ea7-4e98-b11b-712e6bd319e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:27.322266 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf026ca0-2ea7-4e98-b11b-712e6bd319e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:27.323755 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf026ca0-2ea7-4e98-b11b-712e6bd319e9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:27.335517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 633/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 23 11:29:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:27.342176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07191cd-75b7-4d29-b5d1-174f1545ffa0 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:27.441054 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07191cd-75b7-4d29-b5d1-174f1545ffa0 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:27.474326 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f60aaa6-1ba2-4af7-a00e-7799637b734d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:27.474997 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60aaa6-1ba2-4af7-a00e-7799637b734d None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:27.475284 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60aaa6-1ba2-4af7-a00e-7799637b734d None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:27.475560 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60aaa6-1ba2-4af7-a00e-7799637b734d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:27.476380 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f60aaa6-1ba2-4af7-a00e-7799637b734d None admin] RBAC: Adding query filter params (role.id=c9e006de0d6c4812ab29d778e9c8a830, scope.project.id=886c2939a14b48febc2e2fe639c8f1e4, user.id=ac04b32777ca4f929e9e59deee6e0260) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:27.476692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f60aaa6-1ba2-4af7-a00e-7799637b734d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:27.478113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f60aaa6-1ba2-4af7-a00e-7799637b734d None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:27.484719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 654/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 23 11:29:27 2022] GET /identity/v3/role_assignments?user.id=ac04b32777ca4f929e9e59deee6e0260&scope.project.id=886c2939a14b48febc2e2fe639c8f1e4&role.id=c9e006de0d6c4812ab29d778e9c8a830 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:29.544862 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2256928-c23e-497a-8d36-bca1b7914533 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:29.545362 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2256928-c23e-497a-8d36-bca1b7914533 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:29.545447 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2256928-c23e-497a-8d36-bca1b7914533 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:29.546861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 634/1288] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:29:29.551885 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ea117-ecee-44c1-8043-8be0df688fac None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:29.552328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ea117-ecee-44c1-8043-8be0df688fac None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:29.552435 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ea117-ecee-44c1-8043-8be0df688fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:29.569578 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-b21ea117-ecee-44c1-8043-8be0df688fac None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:29:29.570777 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b21ea117-ecee-44c1-8043-8be0df688fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:29:29.599701 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:29.635091 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 655/1289] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:29:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:29:29.658178 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2256928-c23e-497a-8d36-bca1b7914533 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:29.758056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2256928-c23e-497a-8d36-bca1b7914533 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:29.768984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:29.791129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ef951-e85a-4c97-a105-5f040408f6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:29.791985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ef951-e85a-4c97-a105-5f040408f6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:29.792433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ef951-e85a-4c97-a105-5f040408f6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:29.792710 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ef951-e85a-4c97-a105-5f040408f6e3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:29.793361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ef951-e85a-4c97-a105-5f040408f6e3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:29.794739 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ef951-e85a-4c97-a105-5f040408f6e3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:29.806115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-d46ef951-e85a-4c97-a105-5f040408f6e3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 11:29:29.806115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:29.806115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:29:29.806115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:29:29.806115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:29:29.806115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:29:29.806115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:29:29.806115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:29:29.806115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 11:29:29.806115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:29.806115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:29:29.806115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:29.806115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:29.806115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:29.806115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:29.806115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:29.806115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:29.807877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:29.807877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:29.807877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:29.807877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:29.807877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:29.807877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:29.807877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:29:29.807877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:29:29.807877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:29:29.807877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:29:29.807877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:29.807877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:29.807877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:29.807877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:29.807877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:29.807877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:29.809019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:29.809019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:29.809019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:29.809019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:29.809019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:29.809019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:29.809019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:29.809019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:29.809019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:29.809019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:29.809019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:29:29.809019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:29:29.809019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 11:29:29.809019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:29.809019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 635/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jun 23 11:29:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:29.813808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ea117-ecee-44c1-8043-8be0df688fac None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:29.918453 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ea117-ecee-44c1-8043-8be0df688fac None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:29.953263 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769c9ff-fd51-4549-8028-cc76b7ddfef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:29.953976 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769c9ff-fd51-4549-8028-cc76b7ddfef2 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:29.954305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769c9ff-fd51-4549-8028-cc76b7ddfef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:29.954604 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769c9ff-fd51-4549-8028-cc76b7ddfef2 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:29.955299 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769c9ff-fd51-4549-8028-cc76b7ddfef2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:29.955603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769c9ff-fd51-4549-8028-cc76b7ddfef2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:29.956998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769c9ff-fd51-4549-8028-cc76b7ddfef2 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:29.961715 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 656/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Thu Jun 23 11:29:29 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:29.967470 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ef951-e85a-4c97-a105-5f040408f6e3 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:30.067049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ef951-e85a-4c97-a105-5f040408f6e3 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:30.099022 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dde4b7-37a3-409c-93f5-77da83f0cfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:30.099669 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dde4b7-37a3-409c-93f5-77da83f0cfd4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:30.099947 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dde4b7-37a3-409c-93f5-77da83f0cfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:30.100218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dde4b7-37a3-409c-93f5-77da83f0cfd4 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:30.100913 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dde4b7-37a3-409c-93f5-77da83f0cfd4 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:30.101216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dde4b7-37a3-409c-93f5-77da83f0cfd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:30.102588 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dde4b7-37a3-409c-93f5-77da83f0cfd4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:30.107417 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 636/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Thu Jun 23 11:29:29 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:32.128794 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e97531-b597-4d44-8c6f-fb3032235315 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:32.129370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e97531-b597-4d44-8c6f-fb3032235315 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:32.129458 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e97531-b597-4d44-8c6f-fb3032235315 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:32.131124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 657/1293] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:29:32.136759 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f06de-f790-436a-8c09-f6443a1063be None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:32.137193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f06de-f790-436a-8c09-f6443a1063be None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:32.137317 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f06de-f790-436a-8c09-f6443a1063be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:32.155532 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ca9f06de-f790-436a-8c09-f6443a1063be None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:29:32.156724 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ca9f06de-f790-436a-8c09-f6443a1063be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:29:32.186173 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:32.223471 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 637/1294] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:29:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:29:32.246978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e97531-b597-4d44-8c6f-fb3032235315 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:32.278908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:32.348709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e97531-b597-4d44-8c6f-fb3032235315 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:32.382216 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a603d13-4034-4fc9-b13a-d7419d6fc243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:32.382846 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a603d13-4034-4fc9-b13a-d7419d6fc243 None admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:32.383129 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a603d13-4034-4fc9-b13a-d7419d6fc243 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:32.383407 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a603d13-4034-4fc9-b13a-d7419d6fc243 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:32.384124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a603d13-4034-4fc9-b13a-d7419d6fc243 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:32.385997 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a603d13-4034-4fc9-b13a-d7419d6fc243 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:32.500862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 658/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 23 11:29:32 2022] POST /identity/v3/services => generated 246 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:29:34.532850 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036027d-068e-417d-9566-d3822a704a8c None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:34.533353 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036027d-068e-417d-9566-d3822a704a8c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:34.533440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036027d-068e-417d-9566-d3822a704a8c None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:34.534829 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 638/1296] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:29:34.539918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c25a96-ef65-440b-99e7-2e5527f32f5f None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:34.540332 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c25a96-ef65-440b-99e7-2e5527f32f5f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:34.540474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c25a96-ef65-440b-99e7-2e5527f32f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:34.557195 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-b3c25a96-ef65-440b-99e7-2e5527f32f5f None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:29:34.559169 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b3c25a96-ef65-440b-99e7-2e5527f32f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:29:34.628877 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 659/1297] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:29:34 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:29:34.652286 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036027d-068e-417d-9566-d3822a704a8c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:34.754716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036027d-068e-417d-9566-d3822a704a8c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:34.788188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a98021-1289-4739-a92d-cb81ddcd9f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:34.788974 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a98021-1289-4739-a92d-cb81ddcd9f1e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:34.789254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a98021-1289-4739-a92d-cb81ddcd9f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:34.789529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a98021-1289-4739-a92d-cb81ddcd9f1e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:34.790179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a98021-1289-4739-a92d-cb81ddcd9f1e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:34.791578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a98021-1289-4739-a92d-cb81ddcd9f1e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:34.803672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-96a98021-1289-4739-a92d-cb81ddcd9f1e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 11:29:34.803672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:34.803672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:29:34.803672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:29:34.803672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:29:34.803672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:29:34.803672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:29:34.803672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:29:34.803672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 11:29:34.803672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:34.803672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:29:34.803672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:34.803672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:34.803672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:34.803672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:34.803672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:34.803672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:34.804841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:34.804841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:34.804841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:34.804841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:34.804841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:34.804841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:34.804841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:29:34.804841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:29:34.804841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:29:34.804841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:29:34.804841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:34.804841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:34.804841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:34.804841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:34.804841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:34.804841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:34.805885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:34.805885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:34.805885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:34.805885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:34.805885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:34.805885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:34.805885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:34.805885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:34.805885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:34.805885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:34.805885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:29:34.805885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:29:34.805885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 11:29:34.805885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:34.805885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 639/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jun 23 11:29:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:29:34.811034 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c25a96-ef65-440b-99e7-2e5527f32f5f None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:34.829301 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:34.908935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c25a96-ef65-440b-99e7-2e5527f32f5f None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:34.943839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71b1e6-a6ca-426a-bcc7-dffcf7da1dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:34.944521 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71b1e6-a6ca-426a-bcc7-dffcf7da1dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:34.944829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71b1e6-a6ca-426a-bcc7-dffcf7da1dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:34.945109 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71b1e6-a6ca-426a-bcc7-dffcf7da1dc3 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:34.945963 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f71b1e6-a6ca-426a-bcc7-dffcf7da1dc3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:34.946544 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f71b1e6-a6ca-426a-bcc7-dffcf7da1dc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:34.948022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f71b1e6-a6ca-426a-bcc7-dffcf7da1dc3 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:34.953803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 660/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Thu Jun 23 11:29:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:34.961700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a98021-1289-4739-a92d-cb81ddcd9f1e None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:34.992737 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:35.060665 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a98021-1289-4739-a92d-cb81ddcd9f1e None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:35.094650 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18c529-52b7-4bb3-9e00-f17df9bc1bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:35.095309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18c529-52b7-4bb3-9e00-f17df9bc1bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:35.095596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18c529-52b7-4bb3-9e00-f17df9bc1bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:35.095871 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18c529-52b7-4bb3-9e00-f17df9bc1bc0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:35.096664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d18c529-52b7-4bb3-9e00-f17df9bc1bc0 None admin] RBAC: Adding query filter params (interface=public, service_id=629395167d8147258dad1b7502e24464, region_id=RegionOne) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:35.096979 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d18c529-52b7-4bb3-9e00-f17df9bc1bc0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:35.098381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d18c529-52b7-4bb3-9e00-f17df9bc1bc0 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:35.103346 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 640/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jun 23 11:29:34 2022] GET /identity/v3/endpoints?service_id=629395167d8147258dad1b7502e24464&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:29:35.109119 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71b1e6-a6ca-426a-bcc7-dffcf7da1dc3 None admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:35.189186 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:35.206499 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71b1e6-a6ca-426a-bcc7-dffcf7da1dc3 None admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:35.239392 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514596a-c5b5-40fc-8db6-796dda7b9e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:35.240055 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514596a-c5b5-40fc-8db6-796dda7b9e24 None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:35.240339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514596a-c5b5-40fc-8db6-796dda7b9e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:35.240617 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514596a-c5b5-40fc-8db6-796dda7b9e24 None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:35.241297 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514596a-c5b5-40fc-8db6-796dda7b9e24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:35.242701 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514596a-c5b5-40fc-8db6-796dda7b9e24 None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:35.250353 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 661/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jun 23 11:29:35 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:29:37.309038 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd38fb2-fc36-4b7e-a92b-1d7efa980863 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:37.309652 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd38fb2-fc36-4b7e-a92b-1d7efa980863 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:37.309652 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd38fb2-fc36-4b7e-a92b-1d7efa980863 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:37.311016 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 641/1302] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:29:37.316237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0319f48-5939-4a65-973f-dad468662276 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:37.316652 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0319f48-5939-4a65-973f-dad468662276 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:37.316794 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0319f48-5939-4a65-973f-dad468662276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:37.332879 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-d0319f48-5939-4a65-973f-dad468662276 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:29:37.334047 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d0319f48-5939-4a65-973f-dad468662276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:29:37.397944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 662/1303] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:29:37 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:29:37.421487 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd38fb2-fc36-4b7e-a92b-1d7efa980863 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:37.515335 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:37.522431 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd38fb2-fc36-4b7e-a92b-1d7efa980863 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:37.556481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0890372-a0cc-4914-9c53-b42010d62c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:37.557275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0890372-a0cc-4914-9c53-b42010d62c36 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:37.557566 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0890372-a0cc-4914-9c53-b42010d62c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:37.557838 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0890372-a0cc-4914-9c53-b42010d62c36 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:37.558488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0890372-a0cc-4914-9c53-b42010d62c36 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:37.559945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0890372-a0cc-4914-9c53-b42010d62c36 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:37.571777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-d0890372-a0cc-4914-9c53-b42010d62c36 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 11:29:37.571777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:37.571777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:29:37.571777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:29:37.571777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:29:37.571777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:29:37.571777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:29:37.571777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:29:37.571777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 11:29:37.571777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:37.571777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:29:37.571777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:37.571777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:29:37.571777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:29:37.571777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:29:37.571777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:29:37.571777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:29:37.573641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:29:37.573641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:29:37.573641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:29:37.573641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:29:37.573641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:29:37.573641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:29:37.573641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:29:37.573641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:29:37.573641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:29:37.573641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:29:37.573641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:29:37.573641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:29:37.573641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:29:37.573641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:29:37.573641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:29:37.573641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:29:37.574832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:29:37.574832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jun 23 11:29:37.574832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:29:37.574832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:29:37.574832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:29:37.574832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:29:37.574832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:29:37.574832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:29:37.574832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:29:37.574832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:29:37.574832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:29:37.574832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:29:37.574832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 11:29:37.574832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jun 23 11:29:37.578453 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 642/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jun 23 11:29:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:29:37.579510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0319f48-5939-4a65-973f-dad468662276 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:37.655408 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:37.677403 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0319f48-5939-4a65-973f-dad468662276 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:37.710008 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6114a8-fa0f-41f5-a85e-0fcde2d50a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:37.710692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6114a8-fa0f-41f5-a85e-0fcde2d50a0b None admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:37.710981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6114a8-fa0f-41f5-a85e-0fcde2d50a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:37.711259 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6114a8-fa0f-41f5-a85e-0fcde2d50a0b None admin] PATH_INFO: `/v3/services` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:37.711981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6114a8-fa0f-41f5-a85e-0fcde2d50a0b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:29:37.712273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6114a8-fa0f-41f5-a85e-0fcde2d50a0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:37.714210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6114a8-fa0f-41f5-a85e-0fcde2d50a0b None admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:37.719933 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 663/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Thu Jun 23 11:29:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:29:37.725579 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0890372-a0cc-4914-9c53-b42010d62c36 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:37.822256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0890372-a0cc-4914-9c53-b42010d62c36 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:37.855969 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893a9c4-c7b5-49cb-8e2e-76d4ad5deb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:37.856607 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893a9c4-c7b5-49cb-8e2e-76d4ad5deb84 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:37.856886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893a9c4-c7b5-49cb-8e2e-76d4ad5deb84 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:37.857154 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893a9c4-c7b5-49cb-8e2e-76d4ad5deb84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:37.857878 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893a9c4-c7b5-49cb-8e2e-76d4ad5deb84 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:37.859253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893a9c4-c7b5-49cb-8e2e-76d4ad5deb84 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:37.946590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 643/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jun 23 11:29:37 2022] POST /identity/v3/endpoints => generated 331 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:29:47.717174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=30 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:47.941809 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:35:51.835460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0b2c2-f13b-424a-9856-371cf49d6276 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:35:51.836560 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0b2c2-f13b-424a-9856-371cf49d6276 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:35:51.837062 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0b2c2-f13b-424a-9856-371cf49d6276 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:35:51.844239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bdfb0-e4e8-4772-b49e-df23b27107cf None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:35:51.849761 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bdfb0-e4e8-4772-b49e-df23b27107cf None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:35:51.849761 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bdfb0-e4e8-4772-b49e-df23b27107cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:35:51.849961 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 664/1307] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:35:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:35:51.950638 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-882bdfb0-e4e8-4772-b49e-df23b27107cf None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:35:51.952298 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-882bdfb0-e4e8-4772-b49e-df23b27107cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:35:52.173615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 644/1308] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:35:52.235260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6022e54-ef3b-4e06-803c-bfa279f677df None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:35:52.235702 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6022e54-ef3b-4e06-803c-bfa279f677df None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:35:52.236113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6022e54-ef3b-4e06-803c-bfa279f677df None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:35:52.238355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 665/1309] 10.222.0.13 () {54 vars in 964 bytes} [Thu Jun 23 11:35:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:35:52.257646 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e81a3c-9f91-4a2e-b099-3c94b2602273 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:35:52.257936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e81a3c-9f91-4a2e-b099-3c94b2602273 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:35:52.258217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e81a3c-9f91-4a2e-b099-3c94b2602273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:35:52.424198 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-77e81a3c-9f91-4a2e-b099-3c94b2602273 None None] MFA Rules not processed for user `810fa3f54cd04f989252765f7e76e602`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:35:52.427033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-77e81a3c-9f91-4a2e-b099-3c94b2602273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:35:52.478744 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:35:52.586740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 645/1310] 10.222.0.13 () {58 vars in 1061 bytes} [Thu Jun 23 11:35:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:35:52.597281 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6022e54-ef3b-4e06-803c-bfa279f677df None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:35:52.599619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b6022e54-ef3b-4e06-803c-bfa279f677df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:35:52.727190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6022e54-ef3b-4e06-803c-bfa279f677df None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:35:52.809761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e83933-8b7b-4eec-80cc-5784836a3860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:35:52.810767 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e83933-8b7b-4eec-80cc-5784836a3860 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:35:52.811195 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e83933-8b7b-4eec-80cc-5784836a3860 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:35:52.811629 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e83933-8b7b-4eec-80cc-5784836a3860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:35:52.813528 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-33e83933-8b7b-4eec-80cc-5784836a3860 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:35:52.875019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:35:52.937245 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e83933-8b7b-4eec-80cc-5784836a3860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:35:52.939035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e83933-8b7b-4eec-80cc-5784836a3860 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:35:52.961815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 666/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:35:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:35:55.324445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb77eb-d4c3-4f93-9997-51d2ae8d685f None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:35:55.325012 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb77eb-d4c3-4f93-9997-51d2ae8d685f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:35:55.325106 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb77eb-d4c3-4f93-9997-51d2ae8d685f None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:35:55.326563 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 646/1312] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:35:55.332063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245daa9-7a41-446f-9171-fe99cc145d15 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:35:55.332453 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245daa9-7a41-446f-9171-fe99cc145d15 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:35:55.332744 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245daa9-7a41-446f-9171-fe99cc145d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:35:55.403732 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-e245daa9-7a41-446f-9171-fe99cc145d15 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:35:55.404928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e245daa9-7a41-446f-9171-fe99cc145d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:35:55.505663 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 667/1313] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:35:55.544638 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb77eb-d4c3-4f93-9997-51d2ae8d685f None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:35:55.546162 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3aeb77eb-d4c3-4f93-9997-51d2ae8d685f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:35:55.654683 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb77eb-d4c3-4f93-9997-51d2ae8d685f None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:35:55.733083 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:35:55.736007 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ba441-36c1-4d6b-99ce-30f24f2f341a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:35:55.736692 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ba441-36c1-4d6b-99ce-30f24f2f341a service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:35:55.736961 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ba441-36c1-4d6b-99ce-30f24f2f341a service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:35:55.737229 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ba441-36c1-4d6b-99ce-30f24f2f341a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:35:55.739719 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2f0ba441-36c1-4d6b-99ce-30f24f2f341a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:35:55.853418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ba441-36c1-4d6b-99ce-30f24f2f341a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:35:55.854902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ba441-36c1-4d6b-99ce-30f24f2f341a service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:35:55.873906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 647/1314] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:35:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:35:58.415718 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464849b-68da-42c8-b674-3beac3d8d4f4 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:35:58.416303 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464849b-68da-42c8-b674-3beac3d8d4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:35:58.416303 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464849b-68da-42c8-b674-3beac3d8d4f4 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:35:58.417640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 668/1315] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:35:58.422734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77345185-e7e9-47b2-8cbd-a1d62bdb529c None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:35:58.423175 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77345185-e7e9-47b2-8cbd-a1d62bdb529c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:35:58.423289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77345185-e7e9-47b2-8cbd-a1d62bdb529c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:35:58.488544 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-77345185-e7e9-47b2-8cbd-a1d62bdb529c None None] MFA Rules not processed for user `7e575b3126124c04a504aee5827ae3d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:35:58.489788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-77345185-e7e9-47b2-8cbd-a1d62bdb529c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:35:58.543422 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:35:58.590233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 648/1316] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:35:58 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:35:58.630098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464849b-68da-42c8-b674-3beac3d8d4f4 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:35:58.631675 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f464849b-68da-42c8-b674-3beac3d8d4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:35:58.641297 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:35:58.733403 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464849b-68da-42c8-b674-3beac3d8d4f4 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:35:58.806333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0cdcd-a4e9-480a-9313-fba5271d7b41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:35:58.807006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0cdcd-a4e9-480a-9313-fba5271d7b41 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:35:58.807287 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0cdcd-a4e9-480a-9313-fba5271d7b41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:35:58.807559 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0cdcd-a4e9-480a-9313-fba5271d7b41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:35:58.809187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c2f0cdcd-a4e9-480a-9313-fba5271d7b41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:35:58.870490 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:35:58.916505 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f0cdcd-a4e9-480a-9313-fba5271d7b41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:35:58.917945 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f0cdcd-a4e9-480a-9313-fba5271d7b41 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:35:58.936053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 669/1317] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:35:58 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:36:01.764095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2329d-f47f-4def-a80e-18aba51e0a2e None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:01.764599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2329d-f47f-4def-a80e-18aba51e0a2e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:01.764599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2329d-f47f-4def-a80e-18aba51e0a2e None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:01.765917 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 649/1318] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:01.771246 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1c495-8987-47bc-9404-1dbb6248414f None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:01.771624 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1c495-8987-47bc-9404-1dbb6248414f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:01.771806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1c495-8987-47bc-9404-1dbb6248414f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:01.834975 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-3eb1c495-8987-47bc-9404-1dbb6248414f None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:01.836178 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-3eb1c495-8987-47bc-9404-1dbb6248414f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:01.929389 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 670/1319] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:36:01 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:01.967491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2329d-f47f-4def-a80e-18aba51e0a2e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:01.969008 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-fcf2329d-f47f-4def-a80e-18aba51e0a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:02.071261 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2329d-f47f-4def-a80e-18aba51e0a2e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:02.143017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7f6fb-001a-4c33-a46a-ded037afd8d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:02.143678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7f6fb-001a-4c33-a46a-ded037afd8d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:02.143958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7f6fb-001a-4c33-a46a-ded037afd8d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:02.144229 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7f6fb-001a-4c33-a46a-ded037afd8d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:02.145857 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-62a7f6fb-001a-4c33-a46a-ded037afd8d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:02.152973 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:02.252133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7f6fb-001a-4c33-a46a-ded037afd8d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:02.253594 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7f6fb-001a-4c33-a46a-ded037afd8d0 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:02.271629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 650/1320] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:01 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:36:05.300336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548850f1-ff87-49b7-b15f-4670e72a739b None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:05.300914 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548850f1-ff87-49b7-b15f-4670e72a739b None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:05.301010 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548850f1-ff87-49b7-b15f-4670e72a739b None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:05.302418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 671/1321] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:36:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:05.307718 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac3764-0fc8-40c0-b3eb-b249b445bf40 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:05.308141 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac3764-0fc8-40c0-b3eb-b249b445bf40 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:05.308277 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac3764-0fc8-40c0-b3eb-b249b445bf40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:05.372742 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a9ac3764-0fc8-40c0-b3eb-b249b445bf40 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:05.373925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a9ac3764-0fc8-40c0-b3eb-b249b445bf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:05.433680 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:05.473904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 651/1322] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:36:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:05.527298 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1657438-26df-46c2-8424-74a169924598 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:05.527591 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1657438-26df-46c2-8424-74a169924598 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:05.527866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1657438-26df-46c2-8424-74a169924598 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:05.529272 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 672/1323] 10.222.0.13 () {54 vars in 964 bytes} [Thu Jun 23 11:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:05.536705 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2df821-522e-4582-a1b1-f16dcc808fec None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:05.536974 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2df821-522e-4582-a1b1-f16dcc808fec None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:05.537254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2df821-522e-4582-a1b1-f16dcc808fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:05.620464 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ce2df821-522e-4582-a1b1-f16dcc808fec None None] MFA Rules not processed for user `810fa3f54cd04f989252765f7e76e602`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:05.621700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ce2df821-522e-4582-a1b1-f16dcc808fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:05.704514 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 652/1324] 10.222.0.13 () {58 vars in 1061 bytes} [Thu Jun 23 11:36:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:05.714694 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1657438-26df-46c2-8424-74a169924598 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:05.716213 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b1657438-26df-46c2-8424-74a169924598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:05.735091 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:05.818239 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1657438-26df-46c2-8424-74a169924598 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:05.890432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb26ee-bb7d-4340-af57-a45bf1207cc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:05.891036 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb26ee-bb7d-4340-af57-a45bf1207cc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:05.891314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb26ee-bb7d-4340-af57-a45bf1207cc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:05.891581 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb26ee-bb7d-4340-af57-a45bf1207cc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:05.893178 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-78bb26ee-bb7d-4340-af57-a45bf1207cc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:05.965451 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:06.003257 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb26ee-bb7d-4340-af57-a45bf1207cc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:06.004852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb26ee-bb7d-4340-af57-a45bf1207cc7 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:06.024247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 673/1325] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:36:08.369800 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c60b3-6a21-404d-acad-9bbb566d0348 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:08.370341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c60b3-6a21-404d-acad-9bbb566d0348 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:08.370341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c60b3-6a21-404d-acad-9bbb566d0348 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:08.371905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 653/1326] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:08.376953 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb48a91-5c75-49bd-a014-92c441f4ffee None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:08.377370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb48a91-5c75-49bd-a014-92c441f4ffee None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:08.377504 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb48a91-5c75-49bd-a014-92c441f4ffee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:08.449516 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-8bb48a91-5c75-49bd-a014-92c441f4ffee None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:08.452204 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-8bb48a91-5c75-49bd-a014-92c441f4ffee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:08.551186 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 674/1327] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:36:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:08.593380 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c60b3-6a21-404d-acad-9bbb566d0348 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:08.594987 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-419c60b3-6a21-404d-acad-9bbb566d0348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:08.657989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:08.701607 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c60b3-6a21-404d-acad-9bbb566d0348 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:08.775621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64646ee-b98d-4770-8362-7556d60898c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:08.776270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64646ee-b98d-4770-8362-7556d60898c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:08.776553 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64646ee-b98d-4770-8362-7556d60898c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:08.776826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64646ee-b98d-4770-8362-7556d60898c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:08.778464 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b64646ee-b98d-4770-8362-7556d60898c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:08.886334 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:08.888607 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64646ee-b98d-4770-8362-7556d60898c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:08.890081 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64646ee-b98d-4770-8362-7556d60898c5 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:08.908017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 654/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:36:11.041187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825de64b-dec9-41ed-ab5d-7c7864b484eb None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:11.041649 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825de64b-dec9-41ed-ab5d-7c7864b484eb None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:11.041743 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825de64b-dec9-41ed-ab5d-7c7864b484eb None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:11.043153 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 675/1329] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:11.048471 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9ffc1-ac5a-4708-84c8-a0dbcde6a136 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:11.048857 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9ffc1-ac5a-4708-84c8-a0dbcde6a136 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:11.049138 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9ffc1-ac5a-4708-84c8-a0dbcde6a136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:11.115999 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a6d9ffc1-ac5a-4708-84c8-a0dbcde6a136 None None] MFA Rules not processed for user `7e575b3126124c04a504aee5827ae3d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:11.117218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a6d9ffc1-ac5a-4708-84c8-a0dbcde6a136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:11.234388 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 655/1330] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:36:11 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:11.270932 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825de64b-dec9-41ed-ab5d-7c7864b484eb None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:11.272452 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-825de64b-dec9-41ed-ab5d-7c7864b484eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:11.307368 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:11.381364 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825de64b-dec9-41ed-ab5d-7c7864b484eb None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:11.457739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f0ff0-c244-4850-a402-f7fada74dfdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:11.458339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f0ff0-c244-4850-a402-f7fada74dfdb service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:11.458666 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f0ff0-c244-4850-a402-f7fada74dfdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:11.458942 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f0ff0-c244-4850-a402-f7fada74dfdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:11.460561 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-285f0ff0-c244-4850-a402-f7fada74dfdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:11.538440 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:11.570718 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f0ff0-c244-4850-a402-f7fada74dfdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:11.572175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f0ff0-c244-4850-a402-f7fada74dfdb service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:11.592671 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 676/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:11 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:36:14.297524 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ce2ad-841c-4fb0-97bc-4eaf3a4ddf33 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:14.298031 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ce2ad-841c-4fb0-97bc-4eaf3a4ddf33 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:14.298031 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ce2ad-841c-4fb0-97bc-4eaf3a4ddf33 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:14.299418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 656/1332] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:14.305625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3cb7e-b071-43ab-8267-4f521076d830 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:14.306025 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3cb7e-b071-43ab-8267-4f521076d830 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:14.306178 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3cb7e-b071-43ab-8267-4f521076d830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:14.370207 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-17b3cb7e-b071-43ab-8267-4f521076d830 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:14.371444 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-17b3cb7e-b071-43ab-8267-4f521076d830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:14.466523 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 677/1333] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:14.503812 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ce2ad-841c-4fb0-97bc-4eaf3a4ddf33 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:14.505364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-124ce2ad-841c-4fb0-97bc-4eaf3a4ddf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:14.574369 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:14.607531 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ce2ad-841c-4fb0-97bc-4eaf3a4ddf33 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:14.680695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ad2f6-da6c-43a4-8820-4316bf62c8a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:14.681342 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ad2f6-da6c-43a4-8820-4316bf62c8a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:14.681625 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ad2f6-da6c-43a4-8820-4316bf62c8a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:14.681902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ad2f6-da6c-43a4-8820-4316bf62c8a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:14.683592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-027ad2f6-da6c-43a4-8820-4316bf62c8a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:14.788997 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ad2f6-da6c-43a4-8820-4316bf62c8a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:14.790431 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ad2f6-da6c-43a4-8820-4316bf62c8a1 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:14.800830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:14.808905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 657/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:36:16.887050 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6127d9-0add-4f71-8db5-f4306e5567df None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:16.887644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6127d9-0add-4f71-8db5-f4306e5567df None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:16.887644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6127d9-0add-4f71-8db5-f4306e5567df None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:16.888977 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 678/1335] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:16.894866 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19f225-0ed4-41df-8a47-d50ab88d3f99 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:16.895277 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19f225-0ed4-41df-8a47-d50ab88d3f99 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:16.895416 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19f225-0ed4-41df-8a47-d50ab88d3f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:16.959503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-fa19f225-0ed4-41df-8a47-d50ab88d3f99 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:16.960696 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-fa19f225-0ed4-41df-8a47-d50ab88d3f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:17.054748 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 658/1336] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:17.095212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6127d9-0add-4f71-8db5-f4306e5567df None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:17.096993 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-8b6127d9-0add-4f71-8db5-f4306e5567df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:17.135761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:17.197725 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6127d9-0add-4f71-8db5-f4306e5567df None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:17.269264 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f089c-9a10-4c65-8ec7-481f6dbe3f9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:17.269903 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f089c-9a10-4c65-8ec7-481f6dbe3f9c service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:17.270187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f089c-9a10-4c65-8ec7-481f6dbe3f9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:17.270464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f089c-9a10-4c65-8ec7-481f6dbe3f9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:17.272142 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c78f089c-9a10-4c65-8ec7-481f6dbe3f9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:17.353019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:17.379392 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78f089c-9a10-4c65-8ec7-481f6dbe3f9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:17.380824 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78f089c-9a10-4c65-8ec7-481f6dbe3f9c service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:17.398684 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 679/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:36:20.173440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37975e9-4768-42cb-8a16-0f243cc0ec8c None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:20.173990 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37975e9-4768-42cb-8a16-0f243cc0ec8c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:20.173990 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37975e9-4768-42cb-8a16-0f243cc0ec8c None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:20.175518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 659/1338] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:20.180713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1552621-f693-408c-91ef-85803be54b04 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:20.181159 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1552621-f693-408c-91ef-85803be54b04 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:20.181558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1552621-f693-408c-91ef-85803be54b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:20.247575 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-e1552621-f693-408c-91ef-85803be54b04 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:20.248796 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e1552621-f693-408c-91ef-85803be54b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:20.344234 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 680/1339] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:36:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:36:20.381492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37975e9-4768-42cb-8a16-0f243cc0ec8c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:20.383072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b37975e9-4768-42cb-8a16-0f243cc0ec8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:20.462579 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:20.491053 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37975e9-4768-42cb-8a16-0f243cc0ec8c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:20.564722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89f174-241a-47b8-8a63-b346102d96d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:20.565364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89f174-241a-47b8-8a63-b346102d96d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:20.565642 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89f174-241a-47b8-8a63-b346102d96d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:20.566653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89f174-241a-47b8-8a63-b346102d96d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:20.568332 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-8c89f174-241a-47b8-8a63-b346102d96d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:20.673074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c89f174-241a-47b8-8a63-b346102d96d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:20.674419 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c89f174-241a-47b8-8a63-b346102d96d4 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:20.692537 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 660/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:20 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:36:24.848680 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd1976-547f-41d9-b76c-c3f7a463250c None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:24.849236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd1976-547f-41d9-b76c-c3f7a463250c None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:24.849420 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd1976-547f-41d9-b76c-c3f7a463250c None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:24.850879 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 681/1341] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:36:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:24.856536 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34179ef-b0c7-46a7-b93d-d17d675a98c4 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:24.856921 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34179ef-b0c7-46a7-b93d-d17d675a98c4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:24.857136 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34179ef-b0c7-46a7-b93d-d17d675a98c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:24.863508 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:24.936274 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d34179ef-b0c7-46a7-b93d-d17d675a98c4 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:24.937478 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d34179ef-b0c7-46a7-b93d-d17d675a98c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:25.045600 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 661/1342] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:25.085732 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd1976-547f-41d9-b76c-c3f7a463250c None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:25.088098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-07bd1976-547f-41d9-b76c-c3f7a463250c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:25.137946 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:25.183924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd1976-547f-41d9-b76c-c3f7a463250c None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:25.187648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31753c5-862e-4e87-a470-6795c2c5ea48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:25.188304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31753c5-862e-4e87-a470-6795c2c5ea48 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:25.188585 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31753c5-862e-4e87-a470-6795c2c5ea48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:25.188857 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31753c5-862e-4e87-a470-6795c2c5ea48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:25.198221 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31753c5-862e-4e87-a470-6795c2c5ea48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:25.199709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31753c5-862e-4e87-a470-6795c2c5ea48 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:25.209460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 682/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:36:28.319781 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4aa16e-9273-4494-927f-0408395f4a84 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:28.320286 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4aa16e-9273-4494-927f-0408395f4a84 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:28.320286 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4aa16e-9273-4494-927f-0408395f4a84 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:28.321642 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 662/1344] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:28.326998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1146436-5599-4600-a92f-020170d6b35e None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:28.327433 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1146436-5599-4600-a92f-020170d6b35e None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:28.327564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1146436-5599-4600-a92f-020170d6b35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:28.357829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-a1146436-5599-4600-a92f-020170d6b35e None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:28.359063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a1146436-5599-4600-a92f-020170d6b35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:28.435282 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 683/1345] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:36:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:28.474294 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4aa16e-9273-4494-927f-0408395f4a84 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:28.485281 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4aa16e-9273-4494-927f-0408395f4a84 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:28.495862 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d96d3-70bd-49da-a801-7dfcdb9d0b35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:28.496774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d96d3-70bd-49da-a801-7dfcdb9d0b35 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:28.497205 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d96d3-70bd-49da-a801-7dfcdb9d0b35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:28.497760 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d96d3-70bd-49da-a801-7dfcdb9d0b35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:28.510494 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d96d3-70bd-49da-a801-7dfcdb9d0b35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:28.514390 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d96d3-70bd-49da-a801-7dfcdb9d0b35 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:28.522361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 663/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:36:32.929558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c65fde-806e-4df8-a03d-9203dd337065 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:32.929558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c65fde-806e-4df8-a03d-9203dd337065 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:32.929558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c65fde-806e-4df8-a03d-9203dd337065 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:32.929558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 684/1347] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:36:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:36:32.943613 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6daec-17c9-43bd-8edb-62a7edd6a49a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:32.944119 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6daec-17c9-43bd-8edb-62a7edd6a49a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:32.946572 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6daec-17c9-43bd-8edb-62a7edd6a49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:32.965144 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a8d6daec-17c9-43bd-8edb-62a7edd6a49a None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:32.966328 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a8d6daec-17c9-43bd-8edb-62a7edd6a49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:32.978209 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 664/1348] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:33.016928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c65fde-806e-4df8-a03d-9203dd337065 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:33.028202 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c65fde-806e-4df8-a03d-9203dd337065 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:33.032777 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3addae5-4a03-4fa6-83d3-033988f95292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:33.033422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3addae5-4a03-4fa6-83d3-033988f95292 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:33.033710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3addae5-4a03-4fa6-83d3-033988f95292 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:33.033981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3addae5-4a03-4fa6-83d3-033988f95292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:33.044444 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3addae5-4a03-4fa6-83d3-033988f95292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:33.045903 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3addae5-4a03-4fa6-83d3-033988f95292 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:33.056660 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 685/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:36:38.054761 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e166da-e1b0-415f-8a8a-ec226ceb6591 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:38.055261 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e166da-e1b0-415f-8a8a-ec226ceb6591 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:38.055534 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e166da-e1b0-415f-8a8a-ec226ceb6591 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:38.057258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 665/1350] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 23 11:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:38.062162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3bb3b-7087-49be-8a02-ab5e152a6e8e None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:38.062606 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3bb3b-7087-49be-8a02-ab5e152a6e8e None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:38.062760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3bb3b-7087-49be-8a02-ab5e152a6e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:38.148907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-dee3bb3b-7087-49be-8a02-ab5e152a6e8e None None] MFA Rules not processed for user `ac04b32777ca4f929e9e59deee6e0260`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:38.150100 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-dee3bb3b-7087-49be-8a02-ab5e152a6e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:38.179069 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:38.229784 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 686/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 23 11:36:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:42.404151 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61972026-474b-4130-98ab-5f07b405384f None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:42.405145 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61972026-474b-4130-98ab-5f07b405384f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:42.405523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61972026-474b-4130-98ab-5f07b405384f None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:42.407069 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 666/1352] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 23 11:36:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:42.414432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35584e-a193-46ad-8e06-40d07e3d76ce None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:42.414927 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35584e-a193-46ad-8e06-40d07e3d76ce None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:42.415151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35584e-a193-46ad-8e06-40d07e3d76ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:42.435599 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-6c35584e-a193-46ad-8e06-40d07e3d76ce None None] MFA Rules not processed for user `ac04b32777ca4f929e9e59deee6e0260`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:42.436777 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-6c35584e-a193-46ad-8e06-40d07e3d76ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:42.446034 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 687/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 23 11:36:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:42.960149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=72 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:44.037203 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02511c86-489a-474a-92f6-6752b287237c None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:44.037717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02511c86-489a-474a-92f6-6752b287237c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:44.037875 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02511c86-489a-474a-92f6-6752b287237c None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:44.039427 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 667/1354] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 23 11:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:44.044494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e480a48-3e2b-4c89-8771-3873f5aa9d1c None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:44.044910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e480a48-3e2b-4c89-8771-3873f5aa9d1c None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:44.045103 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e480a48-3e2b-4c89-8771-3873f5aa9d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:44.063582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-7e480a48-3e2b-4c89-8771-3873f5aa9d1c None None] MFA Rules not processed for user `ac04b32777ca4f929e9e59deee6e0260`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:44.064768 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-7e480a48-3e2b-4c89-8771-3873f5aa9d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:44.073109 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 688/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 23 11:36:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:47.836989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be4de5-a1ed-4089-85b1-e2637a8cdc23 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:47.837546 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be4de5-a1ed-4089-85b1-e2637a8cdc23 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:47.837546 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be4de5-a1ed-4089-85b1-e2637a8cdc23 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:47.842934 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 668/1356] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:36:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:47.845353 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca62e35-961c-436f-b454-b1ce1e5a2f71 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:47.845780 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca62e35-961c-436f-b454-b1ce1e5a2f71 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:47.846336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca62e35-961c-436f-b454-b1ce1e5a2f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:47.870718 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-3ca62e35-961c-436f-b454-b1ce1e5a2f71 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:47.872135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-3ca62e35-961c-436f-b454-b1ce1e5a2f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:47.882766 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 689/1357] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:47.953312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6eaba-88a1-4ba1-aee9-70eb431ece0c None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:47.953963 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6eaba-88a1-4ba1-aee9-70eb431ece0c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:47.954436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6eaba-88a1-4ba1-aee9-70eb431ece0c None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:47.956745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 669/1358] 10.222.0.13 () {54 vars in 961 bytes} [Thu Jun 23 11:36:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:47.965637 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a329f-17b4-4101-af64-4dcf3c0793b5 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:47.975592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a329f-17b4-4101-af64-4dcf3c0793b5 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:47.975592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a329f-17b4-4101-af64-4dcf3c0793b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:48.022088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-6f5a329f-17b4-4101-af64-4dcf3c0793b5 None None] MFA Rules not processed for user `161c5ed1e47e450290a1f428ee100b68`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:48.023691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-6f5a329f-17b4-4101-af64-4dcf3c0793b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:48.121061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 690/1359] 10.222.0.13 () {58 vars in 1058 bytes} [Thu Jun 23 11:36:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:48.128889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6eaba-88a1-4ba1-aee9-70eb431ece0c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:48.145703 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6eaba-88a1-4ba1-aee9-70eb431ece0c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:48.151230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06d0bf-29da-439b-885d-4df45a4d7d8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:48.151868 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06d0bf-29da-439b-885d-4df45a4d7d8a service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:48.152149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06d0bf-29da-439b-885d-4df45a4d7d8a service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:48.152418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06d0bf-29da-439b-885d-4df45a4d7d8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:48.162695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06d0bf-29da-439b-885d-4df45a4d7d8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:48.164685 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06d0bf-29da-439b-885d-4df45a4d7d8a service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:48.175205 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 670/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:36:50.444727 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddd1fc-8aae-45c1-b433-452e7a74a5b9 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:50.445279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddd1fc-8aae-45c1-b433-452e7a74a5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:50.445279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddd1fc-8aae-45c1-b433-452e7a74a5b9 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:50.446962 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 691/1361] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:50.452086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a18ab-bc28-40f5-82a4-a5a6adebad7b None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:50.452516 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a18ab-bc28-40f5-82a4-a5a6adebad7b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:50.452659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a18ab-bc28-40f5-82a4-a5a6adebad7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:50.472437 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-962a18ab-bc28-40f5-82a4-a5a6adebad7b None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:50.473631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-962a18ab-bc28-40f5-82a4-a5a6adebad7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:50.495178 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 671/1362] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:36:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:50.528560 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddd1fc-8aae-45c1-b433-452e7a74a5b9 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:50.543541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddd1fc-8aae-45c1-b433-452e7a74a5b9 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:50.551367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812260f-a306-4389-9511-9eccfc7f7244 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:50.552016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812260f-a306-4389-9511-9eccfc7f7244 service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:50.552299 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812260f-a306-4389-9511-9eccfc7f7244 service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:50.552575 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812260f-a306-4389-9511-9eccfc7f7244 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:50.562317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812260f-a306-4389-9511-9eccfc7f7244 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:50.567591 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812260f-a306-4389-9511-9eccfc7f7244 service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:50.575324 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 692/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:36:53.100455 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c54609-7a98-4b25-b146-f9f3ccbb167a None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:53.100998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c54609-7a98-4b25-b146-f9f3ccbb167a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:53.100998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c54609-7a98-4b25-b146-f9f3ccbb167a None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:53.102388 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 672/1364] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:53.107679 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f52366-dc1e-472b-ba13-98c5d1781589 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:53.108347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f52366-dc1e-472b-ba13-98c5d1781589 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:53.108347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f52366-dc1e-472b-ba13-98c5d1781589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:53.139267 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-09f52366-dc1e-472b-ba13-98c5d1781589 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:53.140699 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-09f52366-dc1e-472b-ba13-98c5d1781589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:53.150193 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 693/1365] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:53.198184 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777d88a-05e6-4299-98dc-d2207b71fa68 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:53.198470 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777d88a-05e6-4299-98dc-d2207b71fa68 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:53.198782 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777d88a-05e6-4299-98dc-d2207b71fa68 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:53.200142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 673/1366] 10.222.0.13 () {54 vars in 961 bytes} [Thu Jun 23 11:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:53.205251 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640da08-4ede-49e2-ad6c-28e5e30ca018 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:53.205532 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640da08-4ede-49e2-ad6c-28e5e30ca018 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:53.205800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640da08-4ede-49e2-ad6c-28e5e30ca018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:53.224287 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-5640da08-4ede-49e2-ad6c-28e5e30ca018 None None] MFA Rules not processed for user `161c5ed1e47e450290a1f428ee100b68`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:53.225443 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-5640da08-4ede-49e2-ad6c-28e5e30ca018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:53.234392 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 694/1367] 10.222.0.13 () {58 vars in 1058 bytes} [Thu Jun 23 11:36:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:36:53.242661 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5777d88a-05e6-4299-98dc-d2207b71fa68 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:53.257373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5777d88a-05e6-4299-98dc-d2207b71fa68 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:53.262677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49313e34-047d-4a3b-a41d-be551451c9d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:53.263302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49313e34-047d-4a3b-a41d-be551451c9d1 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:53.263582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49313e34-047d-4a3b-a41d-be551451c9d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:53.263854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49313e34-047d-4a3b-a41d-be551451c9d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:53.271877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49313e34-047d-4a3b-a41d-be551451c9d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:53.273439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49313e34-047d-4a3b-a41d-be551451c9d1 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:53.282643 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 674/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:36:55.559886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f5246-4550-49bb-8a44-90e9d2fa717c None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:55.560432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f5246-4550-49bb-8a44-90e9d2fa717c None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:55.560432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f5246-4550-49bb-8a44-90e9d2fa717c None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:55.562093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 695/1369] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:55.568873 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a2a08-0191-4716-afd3-939431918a72 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:55.569348 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a2a08-0191-4716-afd3-939431918a72 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:55.569428 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a2a08-0191-4716-afd3-939431918a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:55.631750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-574a2a08-0191-4716-afd3-939431918a72 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:55.632943 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-574a2a08-0191-4716-afd3-939431918a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:55.640920 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 675/1370] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:55.685444 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f5246-4550-49bb-8a44-90e9d2fa717c None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:55.700452 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f5246-4550-49bb-8a44-90e9d2fa717c None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:55.705892 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e6ff6-9f41-41c5-93d7-94456089971e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:55.707214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e6ff6-9f41-41c5-93d7-94456089971e service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:55.707496 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e6ff6-9f41-41c5-93d7-94456089971e service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:55.707775 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e6ff6-9f41-41c5-93d7-94456089971e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:55.716358 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e6ff6-9f41-41c5-93d7-94456089971e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:55.717920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e6ff6-9f41-41c5-93d7-94456089971e service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:55.727928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 696/1371] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:36:56.182696 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38ac40-ebbc-4d47-ad59-d4e877972a07 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:56.183057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38ac40-ebbc-4d47-ad59-d4e877972a07 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:56.183434 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38ac40-ebbc-4d47-ad59-d4e877972a07 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:56.185126 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 676/1372] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:36:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:56.190445 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd876f96-f825-44e1-8a56-4a4cefa31542 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:56.191460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd876f96-f825-44e1-8a56-4a4cefa31542 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:56.191742 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd876f96-f825-44e1-8a56-4a4cefa31542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:56.210114 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-dd876f96-f825-44e1-8a56-4a4cefa31542 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:56.211334 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-dd876f96-f825-44e1-8a56-4a4cefa31542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:56.225343 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 697/1373] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:56.570865 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28207ae7-87ae-458b-bc17-e3149d0b7752 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:56.570865 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28207ae7-87ae-458b-bc17-e3149d0b7752 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:56.570865 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28207ae7-87ae-458b-bc17-e3149d0b7752 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:56.573026 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 677/1374] 10.222.0.13 () {54 vars in 963 bytes} [Thu Jun 23 11:36:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:56.576812 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335540aa-4615-4940-af85-05456a7b153f None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:56.577264 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335540aa-4615-4940-af85-05456a7b153f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:56.577501 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335540aa-4615-4940-af85-05456a7b153f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:56.585475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:56.630149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-335540aa-4615-4940-af85-05456a7b153f None None] MFA Rules not processed for user `871dd684032c49d09608f0af05b7a765`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:56.631824 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-335540aa-4615-4940-af85-05456a7b153f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:56.695748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 698/1375] 10.222.0.13 () {58 vars in 1060 bytes} [Thu Jun 23 11:36:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:56.703552 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28207ae7-87ae-458b-bc17-e3149d0b7752 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:56.715579 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28207ae7-87ae-458b-bc17-e3149d0b7752 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:56.719172 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471ea39-b0ae-4c60-af8c-8c8fb311aa58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:56.719810 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471ea39-b0ae-4c60-af8c-8c8fb311aa58 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:56.720090 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471ea39-b0ae-4c60-af8c-8c8fb311aa58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:56.720364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471ea39-b0ae-4c60-af8c-8c8fb311aa58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:56.730389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471ea39-b0ae-4c60-af8c-8c8fb311aa58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:56.731891 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471ea39-b0ae-4c60-af8c-8c8fb311aa58 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:56.744060 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 678/1376] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:36:57.966500 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afc187-fb4f-42ff-825e-d32e900656c5 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:57.967076 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afc187-fb4f-42ff-825e-d32e900656c5 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:57.967076 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afc187-fb4f-42ff-825e-d32e900656c5 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:57.968467 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 699/1377] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:36:57.973724 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02b735-1136-4c0e-83d7-087aaa1d564e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:57.974138 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02b735-1136-4c0e-83d7-087aaa1d564e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:57.974358 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02b735-1136-4c0e-83d7-087aaa1d564e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:57.992677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-8f02b735-1136-4c0e-83d7-087aaa1d564e None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:57.993872 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-8f02b735-1136-4c0e-83d7-087aaa1d564e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:58.001838 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 679/1378] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:36:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:36:58.046975 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afc187-fb4f-42ff-825e-d32e900656c5 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:58.061238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afc187-fb4f-42ff-825e-d32e900656c5 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:58.067766 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f13af-52a0-4857-b90a-d154780549b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:58.068419 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f13af-52a0-4857-b90a-d154780549b7 service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:58.068706 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f13af-52a0-4857-b90a-d154780549b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:58.068981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f13af-52a0-4857-b90a-d154780549b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:58.077443 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f13af-52a0-4857-b90a-d154780549b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:58.079076 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f13af-52a0-4857-b90a-d154780549b7 service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:58.089191 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 700/1379] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:00.329991 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c668b-3584-43bc-bc5d-2ba5113970ef None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:00.330573 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c668b-3584-43bc-bc5d-2ba5113970ef None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:00.330573 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c668b-3584-43bc-bc5d-2ba5113970ef None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:00.332619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 680/1380] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:37:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:00.339250 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b12a18-47aa-4bf8-88a0-832fc7245b0c None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:00.339723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b12a18-47aa-4bf8-88a0-832fc7245b0c None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:00.339802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b12a18-47aa-4bf8-88a0-832fc7245b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:00.366062 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-10b12a18-47aa-4bf8-88a0-832fc7245b0c None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:00.367353 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-10b12a18-47aa-4bf8-88a0-832fc7245b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:00.380593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 701/1381] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:00.425354 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c668b-3584-43bc-bc5d-2ba5113970ef None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:00.445728 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c668b-3584-43bc-bc5d-2ba5113970ef None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:00.453011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a024e1cd-442e-405f-a403-3ad6976f12f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:00.453906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024e1cd-442e-405f-a403-3ad6976f12f8 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:00.454394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024e1cd-442e-405f-a403-3ad6976f12f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:00.454996 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024e1cd-442e-405f-a403-3ad6976f12f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:00.466321 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a024e1cd-442e-405f-a403-3ad6976f12f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:00.468219 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a024e1cd-442e-405f-a403-3ad6976f12f8 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:00.480927 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 681/1382] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:02.722246 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603fd52-5ddb-4819-b099-d7293913a7f5 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:02.722246 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603fd52-5ddb-4819-b099-d7293913a7f5 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:02.723672 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603fd52-5ddb-4819-b099-d7293913a7f5 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:02.728739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 702/1383] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:37:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:02.731181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe9ad9-0d9e-4f7c-be15-3792df100574 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:02.731560 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe9ad9-0d9e-4f7c-be15-3792df100574 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:02.731843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe9ad9-0d9e-4f7c-be15-3792df100574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:02.751082 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-23fe9ad9-0d9e-4f7c-be15-3792df100574 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:02.752295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-23fe9ad9-0d9e-4f7c-be15-3792df100574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:02.761019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 682/1384] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:02.802631 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603fd52-5ddb-4819-b099-d7293913a7f5 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:02.817068 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603fd52-5ddb-4819-b099-d7293913a7f5 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:02.822470 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd2c4f-7cad-4623-9dbf-45bd35790a8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:02.823127 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd2c4f-7cad-4623-9dbf-45bd35790a8c service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:02.823409 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd2c4f-7cad-4623-9dbf-45bd35790a8c service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:02.823686 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd2c4f-7cad-4623-9dbf-45bd35790a8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:02.832860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd2c4f-7cad-4623-9dbf-45bd35790a8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:02.834480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd2c4f-7cad-4623-9dbf-45bd35790a8c service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:02.843841 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 703/1385] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:05.012739 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188abab-e8ad-4636-a661-aa8f81bd82ff None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:05.013274 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188abab-e8ad-4636-a661-aa8f81bd82ff None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:05.013460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188abab-e8ad-4636-a661-aa8f81bd82ff None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:05.015278 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 683/1386] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:37:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:05.020923 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641d7d6-d88b-45ad-ac07-2ef9896a04ef None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:05.021363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641d7d6-d88b-45ad-ac07-2ef9896a04ef None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:05.021459 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641d7d6-d88b-45ad-ac07-2ef9896a04ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:05.039381 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-6641d7d6-d88b-45ad-ac07-2ef9896a04ef None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:05.040673 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-6641d7d6-d88b-45ad-ac07-2ef9896a04ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:05.048629 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 704/1387] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:05.098942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188abab-e8ad-4636-a661-aa8f81bd82ff None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:05.113392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188abab-e8ad-4636-a661-aa8f81bd82ff None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:05.118830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802332a-0511-4eac-b416-7b9e09af5c6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:05.119470 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802332a-0511-4eac-b416-7b9e09af5c6d service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:05.119754 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802332a-0511-4eac-b416-7b9e09af5c6d service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:05.120032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802332a-0511-4eac-b416-7b9e09af5c6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:05.128741 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e802332a-0511-4eac-b416-7b9e09af5c6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:05.130363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e802332a-0511-4eac-b416-7b9e09af5c6d service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:05.139773 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 684/1388] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:07.978508 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cc4ee-49d3-4b62-b352-bcd1a64c0a07 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:07.979051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cc4ee-49d3-4b62-b352-bcd1a64c0a07 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:07.979149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cc4ee-49d3-4b62-b352-bcd1a64c0a07 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:07.980519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 705/1389] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:37:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:07.985883 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcba8fe-e947-4841-b87b-cc9b71a828a8 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:07.986370 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcba8fe-e947-4841-b87b-cc9b71a828a8 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:07.986555 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcba8fe-e947-4841-b87b-cc9b71a828a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:08.005200 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-2bcba8fe-e947-4841-b87b-cc9b71a828a8 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:08.006388 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2bcba8fe-e947-4841-b87b-cc9b71a828a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:08.014504 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 685/1390] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:08.061882 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cc4ee-49d3-4b62-b352-bcd1a64c0a07 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:08.078575 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cc4ee-49d3-4b62-b352-bcd1a64c0a07 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:08.085381 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b1330-de8a-4d3c-a2dc-026679e82cca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:08.086032 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b1330-de8a-4d3c-a2dc-026679e82cca service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:08.086316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b1330-de8a-4d3c-a2dc-026679e82cca service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:08.086803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b1330-de8a-4d3c-a2dc-026679e82cca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:08.095934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222b1330-de8a-4d3c-a2dc-026679e82cca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:08.099519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222b1330-de8a-4d3c-a2dc-026679e82cca service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:08.111801 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 706/1391] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:10.312876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683c1b9-92df-403f-8952-d6079a98c0bc None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:10.313405 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683c1b9-92df-403f-8952-d6079a98c0bc None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:10.313405 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683c1b9-92df-403f-8952-d6079a98c0bc None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:10.314777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 686/1392] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:10.320104 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e15d2-e99d-4c60-99be-a31d43d9f3ab None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:10.320539 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e15d2-e99d-4c60-99be-a31d43d9f3ab None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:10.320754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e15d2-e99d-4c60-99be-a31d43d9f3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:10.338999 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-db9e15d2-e99d-4c60-99be-a31d43d9f3ab None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:10.340170 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-db9e15d2-e99d-4c60-99be-a31d43d9f3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:10.348557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 707/1393] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:10.392013 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683c1b9-92df-403f-8952-d6079a98c0bc None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:10.407036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683c1b9-92df-403f-8952-d6079a98c0bc None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:10.413259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb12ed8-066f-4b90-9468-a8cc7aeec8c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:10.413899 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb12ed8-066f-4b90-9468-a8cc7aeec8c6 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:10.414186 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb12ed8-066f-4b90-9468-a8cc7aeec8c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:10.414462 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb12ed8-066f-4b90-9468-a8cc7aeec8c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:10.422998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb12ed8-066f-4b90-9468-a8cc7aeec8c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:10.424596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb12ed8-066f-4b90-9468-a8cc7aeec8c6 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:10.434331 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 687/1394] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:12.170766 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c22a10-7397-4be0-87fd-8d1ebe950767 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:12.171368 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c22a10-7397-4be0-87fd-8d1ebe950767 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:12.171368 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c22a10-7397-4be0-87fd-8d1ebe950767 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:12.172923 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 708/1395] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:12.178000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9ede7-c274-406c-a6c7-9d98117333c6 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:12.178415 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9ede7-c274-406c-a6c7-9d98117333c6 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:12.178581 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9ede7-c274-406c-a6c7-9d98117333c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:12.195918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-41e9ede7-c274-406c-a6c7-9d98117333c6 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:12.197095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-41e9ede7-c274-406c-a6c7-9d98117333c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:12.204784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 688/1396] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:12.261930 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed33b91-a1fb-45c7-8f18-e5c831aa9832 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:12.262241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed33b91-a1fb-45c7-8f18-e5c831aa9832 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:12.262520 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed33b91-a1fb-45c7-8f18-e5c831aa9832 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:12.264028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 709/1397] 10.222.0.13 () {54 vars in 962 bytes} [Thu Jun 23 11:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:12.269133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8807c25-8039-4270-9bf3-a581127b6ba0 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:12.270051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8807c25-8039-4270-9bf3-a581127b6ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:12.270356 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8807c25-8039-4270-9bf3-a581127b6ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:12.327089 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c8807c25-8039-4270-9bf3-a581127b6ba0 None None] MFA Rules not processed for user `b6105838cbf1409587ee7d9e3be554f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:12.328318 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c8807c25-8039-4270-9bf3-a581127b6ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:12.391406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 689/1398] 10.222.0.13 () {58 vars in 1059 bytes} [Thu Jun 23 11:37:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:12.399982 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed33b91-a1fb-45c7-8f18-e5c831aa9832 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:12.411433 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed33b91-a1fb-45c7-8f18-e5c831aa9832 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:12.415046 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64d798-9b2a-4822-bd00-81e545dce034 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6105838cbf1409587ee7d9e3be554f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:12.415776 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64d798-9b2a-4822-bd00-81e545dce034 service glance] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:12.416043 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64d798-9b2a-4822-bd00-81e545dce034 service glance] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:12.416315 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64d798-9b2a-4822-bd00-81e545dce034 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:12.425186 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64d798-9b2a-4822-bd00-81e545dce034 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:12.426866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64d798-9b2a-4822-bd00-81e545dce034 service glance] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:12.436529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 710/1399] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:12.702606 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55162d0-b110-44fd-b349-73b23d9d8b4c None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:12.702903 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55162d0-b110-44fd-b349-73b23d9d8b4c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:12.703187 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55162d0-b110-44fd-b349-73b23d9d8b4c None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:12.704574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 690/1400] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:12.710430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc12a6-07a1-4e92-a1af-27eb8074ccb6 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:12.710759 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc12a6-07a1-4e92-a1af-27eb8074ccb6 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:12.711037 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc12a6-07a1-4e92-a1af-27eb8074ccb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:12.730088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-e2fc12a6-07a1-4e92-a1af-27eb8074ccb6 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:12.731307 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e2fc12a6-07a1-4e92-a1af-27eb8074ccb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:12.739749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 711/1401] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:12.784069 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55162d0-b110-44fd-b349-73b23d9d8b4c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:12.798907 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55162d0-b110-44fd-b349-73b23d9d8b4c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:12.804913 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1981d-f5d5-4239-9fbd-144996701986 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:12.805546 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1981d-f5d5-4239-9fbd-144996701986 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:12.805832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1981d-f5d5-4239-9fbd-144996701986 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:12.806114 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1981d-f5d5-4239-9fbd-144996701986 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:12.814234 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1981d-f5d5-4239-9fbd-144996701986 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:12.815867 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1981d-f5d5-4239-9fbd-144996701986 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:12.826074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 691/1402] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:13.324085 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51ce6c-0381-4563-ad91-fd3dac9ce843 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:13.324564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51ce6c-0381-4563-ad91-fd3dac9ce843 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:13.324802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51ce6c-0381-4563-ad91-fd3dac9ce843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:13.372976 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-3d51ce6c-0381-4563-ad91-fd3dac9ce843 None None] MFA Rules not processed for user `030614ef56f143f8be6fcc0445ec9568`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:13.374172 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-3d51ce6c-0381-4563-ad91-fd3dac9ce843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:13.407325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:37:13.439374 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 712/1403] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 23 11:37:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:13.481850 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30a2fa-733f-4953-a7d6-4ab8eb90ea6b None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:13.482342 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30a2fa-733f-4953-a7d6-4ab8eb90ea6b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:13.482613 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30a2fa-733f-4953-a7d6-4ab8eb90ea6b None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:13.484412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 692/1404] 10.222.0.13 () {54 vars in 942 bytes} [Thu Jun 23 11:37:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:13.492511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cd564-2923-42dc-8b89-d6fd84cbda74 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:13.492801 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cd564-2923-42dc-8b89-d6fd84cbda74 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:13.493080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cd564-2923-42dc-8b89-d6fd84cbda74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:13.542540 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-3a5cd564-2923-42dc-8b89-d6fd84cbda74 None None] MFA Rules not processed for user `450b1fad96604fb09d0705895c6d3da4`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:13.543781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-3a5cd564-2923-42dc-8b89-d6fd84cbda74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:13.614592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 713/1405] 10.222.0.13 () {58 vars in 1039 bytes} [Thu Jun 23 11:37:13 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:13.623812 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30a2fa-733f-4953-a7d6-4ab8eb90ea6b None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:13.631784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:37:13.638103 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30a2fa-733f-4953-a7d6-4ab8eb90ea6b None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:13.643392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203b421-81c9-465a-a329-3ff96735a042 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450b1fad96604fb09d0705895c6d3da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:13.644552 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203b421-81c9-465a-a329-3ff96735a042 service swift] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:13.645134 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203b421-81c9-465a-a329-3ff96735a042 service swift] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:13.645237 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203b421-81c9-465a-a329-3ff96735a042 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:13.653618 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4203b421-81c9-465a-a329-3ff96735a042 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:13.655243 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4203b421-81c9-465a-a329-3ff96735a042 service swift] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:13.664735 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 693/1406] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 23 11:37:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:37:15.095351 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda89dbc-79b4-4f81-9e4b-3b045f594f25 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:15.095896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda89dbc-79b4-4f81-9e4b-3b045f594f25 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:15.095896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda89dbc-79b4-4f81-9e4b-3b045f594f25 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:15.098597 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 714/1407] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:37:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:15.103382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642847ba-c39e-44ae-9ab3-499940b26ec6 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:15.103785 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642847ba-c39e-44ae-9ab3-499940b26ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:15.103946 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642847ba-c39e-44ae-9ab3-499940b26ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:15.123190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-642847ba-c39e-44ae-9ab3-499940b26ec6 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:15.124392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-642847ba-c39e-44ae-9ab3-499940b26ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:15.132496 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 694/1408] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:15.182294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda89dbc-79b4-4f81-9e4b-3b045f594f25 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:15.197430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda89dbc-79b4-4f81-9e4b-3b045f594f25 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:15.202736 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ddec6-b0ee-4987-bc75-b09c6bf18704 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:15.206349 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ddec6-b0ee-4987-bc75-b09c6bf18704 service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:15.206349 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ddec6-b0ee-4987-bc75-b09c6bf18704 service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:15.206349 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ddec6-b0ee-4987-bc75-b09c6bf18704 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:15.216409 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ddec6-b0ee-4987-bc75-b09c6bf18704 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:15.218047 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ddec6-b0ee-4987-bc75-b09c6bf18704 service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:15.230703 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 715/1409] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:25.117670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=8 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:37:25.224676 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=62 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:37:36.802521 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a52054-35d9-458e-9879-d6acda533220 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:36.803092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a52054-35d9-458e-9879-d6acda533220 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:36.803092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a52054-35d9-458e-9879-d6acda533220 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:36.804450 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 695/1410] 10.222.0.13 () {54 vars in 918 bytes} [Thu Jun 23 11:37:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:36.809446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249e204-5c0c-4003-949f-bb256c1fda9b None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:36.809880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249e204-5c0c-4003-949f-bb256c1fda9b None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:36.809994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249e204-5c0c-4003-949f-bb256c1fda9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:36.827668 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-0249e204-5c0c-4003-949f-bb256c1fda9b None None] MFA Rules not processed for user `ac04b32777ca4f929e9e59deee6e0260`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:36.828852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-0249e204-5c0c-4003-949f-bb256c1fda9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:36.837642 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 716/1411] 10.222.0.13 () {58 vars in 1015 bytes} [Thu Jun 23 11:37:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:36.853069 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d481f2f-b539-499a-abe8-1acce1a98468 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:36.853358 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d481f2f-b539-499a-abe8-1acce1a98468 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:36.853632 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d481f2f-b539-499a-abe8-1acce1a98468 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:36.855032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 696/1412] 10.222.0.13 () {54 vars in 918 bytes} [Thu Jun 23 11:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:36.859579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ab8f7-7332-4d12-9ef3-5b2ed5fb9198 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:36.859871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ab8f7-7332-4d12-9ef3-5b2ed5fb9198 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:36.860148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ab8f7-7332-4d12-9ef3-5b2ed5fb9198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:36.878031 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-568ab8f7-7332-4d12-9ef3-5b2ed5fb9198 None None] MFA Rules not processed for user `161c5ed1e47e450290a1f428ee100b68`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:36.879296 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-568ab8f7-7332-4d12-9ef3-5b2ed5fb9198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:36.890231 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 717/1413] 10.222.0.13 () {58 vars in 1015 bytes} [Thu Jun 23 11:37:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:39.384071 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddabfb-19ff-469f-8675-38ab5cd14491 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:39.384596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddabfb-19ff-469f-8675-38ab5cd14491 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:39.384713 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddabfb-19ff-469f-8675-38ab5cd14491 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:39.386057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 697/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:39.391139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068af8e1-d4d1-4822-8c3e-2ebf9171c3a3 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:39.391556 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068af8e1-d4d1-4822-8c3e-2ebf9171c3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:39.391692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068af8e1-d4d1-4822-8c3e-2ebf9171c3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:39.411137 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-068af8e1-d4d1-4822-8c3e-2ebf9171c3a3 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:39.412302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-068af8e1-d4d1-4822-8c3e-2ebf9171c3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:39.419817 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 718/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:37:39.463722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddabfb-19ff-469f-8675-38ab5cd14491 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:39.478876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddabfb-19ff-469f-8675-38ab5cd14491 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:39.484317 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4597b01-bb5e-4a04-be6e-b0e209101fcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:39.484966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4597b01-bb5e-4a04-be6e-b0e209101fcc service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:39.485250 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4597b01-bb5e-4a04-be6e-b0e209101fcc service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:39.485527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4597b01-bb5e-4a04-be6e-b0e209101fcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:39.494327 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4597b01-bb5e-4a04-be6e-b0e209101fcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:39.495950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4597b01-bb5e-4a04-be6e-b0e209101fcc service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:39.505370 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 698/1416] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:40.411901 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e707f-f4e8-4fb6-866e-d8d6bfa7fdb9 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:40.412443 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e707f-f4e8-4fb6-866e-d8d6bfa7fdb9 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:40.412443 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e707f-f4e8-4fb6-866e-d8d6bfa7fdb9 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:40.413809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 719/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:37:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:40.418784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77848bde-e35d-4926-aa4e-3d371cb63b7a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:40.419211 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77848bde-e35d-4926-aa4e-3d371cb63b7a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:40.419335 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77848bde-e35d-4926-aa4e-3d371cb63b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:40.437750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-77848bde-e35d-4926-aa4e-3d371cb63b7a None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:40.438972 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-77848bde-e35d-4926-aa4e-3d371cb63b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:40.446909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 699/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:40.600401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e707f-f4e8-4fb6-866e-d8d6bfa7fdb9 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:40.615424 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e707f-f4e8-4fb6-866e-d8d6bfa7fdb9 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:40.620787 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4818483-9862-4ec5-a96b-92c9872a524c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:40.621411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4818483-9862-4ec5-a96b-92c9872a524c service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:40.621689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4818483-9862-4ec5-a96b-92c9872a524c service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:40.621960 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4818483-9862-4ec5-a96b-92c9872a524c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:40.630387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4818483-9862-4ec5-a96b-92c9872a524c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:40.632105 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4818483-9862-4ec5-a96b-92c9872a524c service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:40.641466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 720/1419] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:41.525479 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429c834-2fda-477b-a2ea-14a3d16506d7 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:41.526011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429c834-2fda-477b-a2ea-14a3d16506d7 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:41.526126 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429c834-2fda-477b-a2ea-14a3d16506d7 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:41.527467 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 700/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:41.532268 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e467a-c0bc-4505-9384-5ec0e98468d1 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:41.532696 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e467a-c0bc-4505-9384-5ec0e98468d1 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:41.532826 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e467a-c0bc-4505-9384-5ec0e98468d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:41.551869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-587e467a-c0bc-4505-9384-5ec0e98468d1 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:41.553078 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-587e467a-c0bc-4505-9384-5ec0e98468d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:41.560685 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 721/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:41.715358 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429c834-2fda-477b-a2ea-14a3d16506d7 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:41.730455 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429c834-2fda-477b-a2ea-14a3d16506d7 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:41.735871 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee666c-8842-4c62-9b97-4af50c294813 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:41.736504 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee666c-8842-4c62-9b97-4af50c294813 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:41.736787 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee666c-8842-4c62-9b97-4af50c294813 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:41.737061 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee666c-8842-4c62-9b97-4af50c294813 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:41.745253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee666c-8842-4c62-9b97-4af50c294813 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:41.746807 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee666c-8842-4c62-9b97-4af50c294813 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:41.756139 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 701/1422] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:42.612492 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d5dde-74db-44ef-9c6a-b9a7927e25d8 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:42.613059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d5dde-74db-44ef-9c6a-b9a7927e25d8 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:42.613059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d5dde-74db-44ef-9c6a-b9a7927e25d8 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:42.615043 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 722/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:37:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:42.619883 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036e7b8-e99f-4fcc-833d-9d69cb51b7f5 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:42.620330 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036e7b8-e99f-4fcc-833d-9d69cb51b7f5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:42.620426 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036e7b8-e99f-4fcc-833d-9d69cb51b7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:42.638889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-9036e7b8-e99f-4fcc-833d-9d69cb51b7f5 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:42.640084 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9036e7b8-e99f-4fcc-833d-9d69cb51b7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:42.648482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 702/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:42.795859 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d5dde-74db-44ef-9c6a-b9a7927e25d8 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:42.810064 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d5dde-74db-44ef-9c6a-b9a7927e25d8 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:42.815589 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d64562-0d2b-45a0-a22a-3d8d4784a7e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:42.816226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d64562-0d2b-45a0-a22a-3d8d4784a7e0 service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:42.816508 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d64562-0d2b-45a0-a22a-3d8d4784a7e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:42.817425 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d64562-0d2b-45a0-a22a-3d8d4784a7e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:42.829900 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d64562-0d2b-45a0-a22a-3d8d4784a7e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:42.833336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d64562-0d2b-45a0-a22a-3d8d4784a7e0 service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:42.843022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 723/1425] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:43.704575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77f768-ed4b-4091-b83f-b2b3d80fe117 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:43.705125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77f768-ed4b-4091-b83f-b2b3d80fe117 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:43.705219 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77f768-ed4b-4091-b83f-b2b3d80fe117 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:43.706588 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 703/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:43.712371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff1233-b1cd-4486-9c45-1b6c355fdd38 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:43.712796 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff1233-b1cd-4486-9c45-1b6c355fdd38 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:43.712933 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff1233-b1cd-4486-9c45-1b6c355fdd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:43.730868 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-f5ff1233-b1cd-4486-9c45-1b6c355fdd38 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:43.732052 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f5ff1233-b1cd-4486-9c45-1b6c355fdd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:43.740585 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 724/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:43.891901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77f768-ed4b-4091-b83f-b2b3d80fe117 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:43.906260 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77f768-ed4b-4091-b83f-b2b3d80fe117 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:43.912290 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8f04c-7ef9-4df1-8859-7aa4322405b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:43.912651 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8f04c-7ef9-4df1-8859-7aa4322405b3 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:43.912939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8f04c-7ef9-4df1-8859-7aa4322405b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:43.913216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8f04c-7ef9-4df1-8859-7aa4322405b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:43.921590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d8f04c-7ef9-4df1-8859-7aa4322405b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:43.923210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d8f04c-7ef9-4df1-8859-7aa4322405b3 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:43.933611 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 704/1428] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:44.822047 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f0cc2-e192-40ea-8d6d-efca082636b3 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:44.822606 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f0cc2-e192-40ea-8d6d-efca082636b3 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:44.822824 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f0cc2-e192-40ea-8d6d-efca082636b3 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:44.824041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 725/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:37:44.829078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45db16a-8aaf-43ee-b112-82d05986258a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:44.829510 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45db16a-8aaf-43ee-b112-82d05986258a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:44.829633 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45db16a-8aaf-43ee-b112-82d05986258a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:44.849243 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f45db16a-8aaf-43ee-b112-82d05986258a None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:44.850428 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f45db16a-8aaf-43ee-b112-82d05986258a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:44.858243 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 705/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:45.008758 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f0cc2-e192-40ea-8d6d-efca082636b3 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:45.023346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f0cc2-e192-40ea-8d6d-efca082636b3 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:45.028642 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16db51a4-f763-4901-90af-6a3f97c49cfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:45.029259 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db51a4-f763-4901-90af-6a3f97c49cfc service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:45.029545 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db51a4-f763-4901-90af-6a3f97c49cfc service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:45.029815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db51a4-f763-4901-90af-6a3f97c49cfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:45.037746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16db51a4-f763-4901-90af-6a3f97c49cfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:45.039373 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16db51a4-f763-4901-90af-6a3f97c49cfc service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:45.048522 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 726/1431] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:45.883335 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7306e85-0fa2-4dfa-9b84-8a772bfd7465 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:45.883918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7306e85-0fa2-4dfa-9b84-8a772bfd7465 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:45.883918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7306e85-0fa2-4dfa-9b84-8a772bfd7465 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:45.885246 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 706/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:45.890230 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254906e-b6c3-4f7d-87e5-18375a5a34f4 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:45.890670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254906e-b6c3-4f7d-87e5-18375a5a34f4 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:45.890859 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254906e-b6c3-4f7d-87e5-18375a5a34f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:45.908162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-a254906e-b6c3-4f7d-87e5-18375a5a34f4 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:45.909344 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a254906e-b6c3-4f7d-87e5-18375a5a34f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:45.917720 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 727/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:46.069646 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7306e85-0fa2-4dfa-9b84-8a772bfd7465 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:46.084874 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7306e85-0fa2-4dfa-9b84-8a772bfd7465 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:46.090089 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdfb658-5931-4da5-a624-016cfdfc4cb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:46.090750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdfb658-5931-4da5-a624-016cfdfc4cb2 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:46.091029 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdfb658-5931-4da5-a624-016cfdfc4cb2 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:46.091306 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdfb658-5931-4da5-a624-016cfdfc4cb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:46.100024 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdfb658-5931-4da5-a624-016cfdfc4cb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:46.101591 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdfb658-5931-4da5-a624-016cfdfc4cb2 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:46.110875 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 707/1434] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:47.006048 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63311234-0087-4a22-babe-2be2f7348ab6 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:47.006672 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63311234-0087-4a22-babe-2be2f7348ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:47.006672 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63311234-0087-4a22-babe-2be2f7348ab6 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:47.008064 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 728/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:37:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:47.013019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088be66c-c7bc-4263-92be-dbdf75e13f04 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:47.013420 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088be66c-c7bc-4263-92be-dbdf75e13f04 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:47.013586 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088be66c-c7bc-4263-92be-dbdf75e13f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:47.031283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-088be66c-c7bc-4263-92be-dbdf75e13f04 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:47.032476 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-088be66c-c7bc-4263-92be-dbdf75e13f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:47.040228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 708/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:37:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:47.190899 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63311234-0087-4a22-babe-2be2f7348ab6 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:47.205361 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63311234-0087-4a22-babe-2be2f7348ab6 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:47.211085 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f9d3b-0a1b-47d2-9102-f225143f9ccc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:47.211721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f9d3b-0a1b-47d2-9102-f225143f9ccc service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:47.212006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f9d3b-0a1b-47d2-9102-f225143f9ccc service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:47.212286 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f9d3b-0a1b-47d2-9102-f225143f9ccc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:47.220309 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f9d3b-0a1b-47d2-9102-f225143f9ccc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:47.221895 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f9d3b-0a1b-47d2-9102-f225143f9ccc service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:47.231959 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 729/1437] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:48.090101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a795a-caef-46fd-89b4-5bc0d0ed246e None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:48.090635 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a795a-caef-46fd-89b4-5bc0d0ed246e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:48.090745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a795a-caef-46fd-89b4-5bc0d0ed246e None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:48.092065 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 709/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:48.097342 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89491ed8-2f9d-440e-a19e-7f916c8e9e67 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:48.097799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89491ed8-2f9d-440e-a19e-7f916c8e9e67 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:48.097911 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89491ed8-2f9d-440e-a19e-7f916c8e9e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:48.116634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-89491ed8-2f9d-440e-a19e-7f916c8e9e67 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:48.117847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-89491ed8-2f9d-440e-a19e-7f916c8e9e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:48.125560 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 730/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:37:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:48.277250 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a795a-caef-46fd-89b4-5bc0d0ed246e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:48.292270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a795a-caef-46fd-89b4-5bc0d0ed246e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:48.297492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796ff92-b4c8-4a3b-ae48-f83aee1f0959 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:48.298122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796ff92-b4c8-4a3b-ae48-f83aee1f0959 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:48.298403 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796ff92-b4c8-4a3b-ae48-f83aee1f0959 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:48.298715 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796ff92-b4c8-4a3b-ae48-f83aee1f0959 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:48.307694 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796ff92-b4c8-4a3b-ae48-f83aee1f0959 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:48.309273 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796ff92-b4c8-4a3b-ae48-f83aee1f0959 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:48.319189 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 710/1440] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:50.257290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc2f00-f17c-4879-9650-1c5d523fd50e None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:50.257906 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc2f00-f17c-4879-9650-1c5d523fd50e None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:50.257906 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc2f00-f17c-4879-9650-1c5d523fd50e None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:50.259260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 731/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:50.264480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39119b60-e0fa-42a2-b246-feb99e4549db None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:50.264924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39119b60-e0fa-42a2-b246-feb99e4549db None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:50.265033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39119b60-e0fa-42a2-b246-feb99e4549db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:50.283622 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-39119b60-e0fa-42a2-b246-feb99e4549db None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:50.284807 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-39119b60-e0fa-42a2-b246-feb99e4549db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:50.292601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 711/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:37:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:50.331360 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc2f00-f17c-4879-9650-1c5d523fd50e None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:50.342580 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc2f00-f17c-4879-9650-1c5d523fd50e None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:50.345972 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c230e3-bc08-44c5-995a-1c136d34c696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:50.346626 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c230e3-bc08-44c5-995a-1c136d34c696 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:50.346904 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c230e3-bc08-44c5-995a-1c136d34c696 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:50.347175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c230e3-bc08-44c5-995a-1c136d34c696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:50.356662 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c230e3-bc08-44c5-995a-1c136d34c696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:50.358123 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c230e3-bc08-44c5-995a-1c136d34c696 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:50.367607 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 732/1443] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:52.558495 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2402be-7be8-46e4-b6f1-4d0f47d2ee98 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:52.559067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2402be-7be8-46e4-b6f1-4d0f47d2ee98 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:52.559067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2402be-7be8-46e4-b6f1-4d0f47d2ee98 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:52.560457 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 712/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:52.565964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35569e72-d638-4472-94ae-065452800912 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:52.566463 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35569e72-d638-4472-94ae-065452800912 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:52.566689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35569e72-d638-4472-94ae-065452800912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:52.583933 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-35569e72-d638-4472-94ae-065452800912 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:52.591173 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-35569e72-d638-4472-94ae-065452800912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:52.593329 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 733/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:52.633608 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2402be-7be8-46e4-b6f1-4d0f47d2ee98 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:52.635337 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-da2402be-7be8-46e4-b6f1-4d0f47d2ee98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:52.648461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2402be-7be8-46e4-b6f1-4d0f47d2ee98 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:52.652801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359deac3-b068-41cc-be7b-68120dfae654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:52.652801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359deac3-b068-41cc-be7b-68120dfae654 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:52.653002 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359deac3-b068-41cc-be7b-68120dfae654 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:52.653221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359deac3-b068-41cc-be7b-68120dfae654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:52.662707 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359deac3-b068-41cc-be7b-68120dfae654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:52.664196 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359deac3-b068-41cc-be7b-68120dfae654 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:52.673587 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 713/1446] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:37:55.166852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d3e21-d340-43ab-990a-6979bb59d5b2 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:55.167389 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d3e21-d340-43ab-990a-6979bb59d5b2 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:55.167389 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d3e21-d340-43ab-990a-6979bb59d5b2 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:55.168786 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 734/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:55.173965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05050318-e4e6-4c7d-ad61-2ac9c8eb6077 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:55.174388 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05050318-e4e6-4c7d-ad61-2ac9c8eb6077 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:55.174546 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05050318-e4e6-4c7d-ad61-2ac9c8eb6077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:55.193105 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-05050318-e4e6-4c7d-ad61-2ac9c8eb6077 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:55.194300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-05050318-e4e6-4c7d-ad61-2ac9c8eb6077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:55.202148 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 714/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:37:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:55.241141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d3e21-d340-43ab-990a-6979bb59d5b2 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:55.251671 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d3e21-d340-43ab-990a-6979bb59d5b2 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:55.255727 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084ef93-ead1-4a47-8369-2cc7dd8600c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:55.256351 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084ef93-ead1-4a47-8369-2cc7dd8600c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:55.256631 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084ef93-ead1-4a47-8369-2cc7dd8600c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:55.256906 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084ef93-ead1-4a47-8369-2cc7dd8600c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:55.265528 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084ef93-ead1-4a47-8369-2cc7dd8600c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:55.267003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084ef93-ead1-4a47-8369-2cc7dd8600c4 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:55.276149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 735/1449] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:57.442812 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953c67c-807e-4275-baef-df327933687c None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:57.443382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953c67c-807e-4275-baef-df327933687c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:57.443382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953c67c-807e-4275-baef-df327933687c None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:57.444745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 715/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:37:57.450126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71135e7-ddb9-43fc-806e-a278df2eeef8 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:57.450565 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71135e7-ddb9-43fc-806e-a278df2eeef8 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:57.450713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71135e7-ddb9-43fc-806e-a278df2eeef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:57.468486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-a71135e7-ddb9-43fc-806e-a278df2eeef8 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:57.469661 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a71135e7-ddb9-43fc-806e-a278df2eeef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:57.477880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 736/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:37:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:57.515663 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953c67c-807e-4275-baef-df327933687c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:57.526698 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953c67c-807e-4275-baef-df327933687c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:57.530116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516ae77-3d8d-47f8-8bd1-8f3a6d57de41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:57.530787 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516ae77-3d8d-47f8-8bd1-8f3a6d57de41 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:57.531071 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516ae77-3d8d-47f8-8bd1-8f3a6d57de41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:57.531348 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516ae77-3d8d-47f8-8bd1-8f3a6d57de41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:57.540744 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3516ae77-3d8d-47f8-8bd1-8f3a6d57de41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:57.542169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3516ae77-3d8d-47f8-8bd1-8f3a6d57de41 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:57.551397 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 716/1452] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:59.716587 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10d674-4e57-4633-9c14-2aa0c70670bc None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:59.717124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10d674-4e57-4633-9c14-2aa0c70670bc None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:59.717124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10d674-4e57-4633-9c14-2aa0c70670bc None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:59.718556 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 737/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:37:59.725103 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9cee8-3bd1-46ca-8f24-8b0a2863b56f None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:59.725528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9cee8-3bd1-46ca-8f24-8b0a2863b56f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:59.725655 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9cee8-3bd1-46ca-8f24-8b0a2863b56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:59.743472 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c5f9cee8-3bd1-46ca-8f24-8b0a2863b56f None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:59.744613 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c5f9cee8-3bd1-46ca-8f24-8b0a2863b56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:59.752283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 717/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:59.791151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10d674-4e57-4633-9c14-2aa0c70670bc None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:59.802349 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10d674-4e57-4633-9c14-2aa0c70670bc None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:59.805830 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424384b-dabc-4032-80e1-368eeb212fbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:59.815466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424384b-dabc-4032-80e1-368eeb212fbc service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:59.815466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424384b-dabc-4032-80e1-368eeb212fbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:59.815466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424384b-dabc-4032-80e1-368eeb212fbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:59.816865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424384b-dabc-4032-80e1-368eeb212fbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:59.818788 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424384b-dabc-4032-80e1-368eeb212fbc service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:59.828116 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 738/1455] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:38:02.028163 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abcea8-6a8b-4d29-8a8b-8c33dfd4f002 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:02.028735 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abcea8-6a8b-4d29-8a8b-8c33dfd4f002 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:02.028735 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abcea8-6a8b-4d29-8a8b-8c33dfd4f002 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:02.030115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 718/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:38:02.036133 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec352add-b1ec-4d64-8148-580b9cdc0e10 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:02.036548 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec352add-b1ec-4d64-8148-580b9cdc0e10 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:02.036700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec352add-b1ec-4d64-8148-580b9cdc0e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:02.053958 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-ec352add-b1ec-4d64-8148-580b9cdc0e10 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:38:02.055177 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-ec352add-b1ec-4d64-8148-580b9cdc0e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:02.062845 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 739/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:38:02.101181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abcea8-6a8b-4d29-8a8b-8c33dfd4f002 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:02.111953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abcea8-6a8b-4d29-8a8b-8c33dfd4f002 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:02.115881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b26d3-fded-4398-8d19-9d93623a892c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:02.116514 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b26d3-fded-4398-8d19-9d93623a892c service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:02.116798 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b26d3-fded-4398-8d19-9d93623a892c service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:02.117071 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b26d3-fded-4398-8d19-9d93623a892c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:02.125851 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b26d3-fded-4398-8d19-9d93623a892c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:02.127300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b26d3-fded-4398-8d19-9d93623a892c service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:02.137230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 719/1458] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:38:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:38:04.216721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8773b06-8bbb-4778-b62c-1dd214449436 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:04.217271 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8773b06-8bbb-4778-b62c-1dd214449436 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:04.217271 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8773b06-8bbb-4778-b62c-1dd214449436 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:04.218741 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 740/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:38:04.224772 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbfbc1-fb4d-4ae3-bffb-151303f4d942 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:04.225180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbfbc1-fb4d-4ae3-bffb-151303f4d942 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:04.225342 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbfbc1-fb4d-4ae3-bffb-151303f4d942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:04.242694 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-2dbbfbc1-fb4d-4ae3-bffb-151303f4d942 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:38:04.243889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2dbbfbc1-fb4d-4ae3-bffb-151303f4d942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:04.251883 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 720/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:38:04.289995 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8773b06-8bbb-4778-b62c-1dd214449436 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:04.300635 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8773b06-8bbb-4778-b62c-1dd214449436 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:04.304290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac620ec2-c014-4aca-a9ab-39fde64c7e07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:04.304915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac620ec2-c014-4aca-a9ab-39fde64c7e07 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:04.305204 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac620ec2-c014-4aca-a9ab-39fde64c7e07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:04.305478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac620ec2-c014-4aca-a9ab-39fde64c7e07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:04.314635 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac620ec2-c014-4aca-a9ab-39fde64c7e07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:04.316071 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac620ec2-c014-4aca-a9ab-39fde64c7e07 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:04.325935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 741/1461] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:38:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:38:06.417879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb24773-b6c3-4b10-a18a-12b2a179ef4f None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:06.418391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb24773-b6c3-4b10-a18a-12b2a179ef4f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:06.418391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb24773-b6c3-4b10-a18a-12b2a179ef4f None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:06.419774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 721/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:38:06.425141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6cbbd9-83b3-4878-87a0-00118f19038a None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:06.426054 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6cbbd9-83b3-4878-87a0-00118f19038a None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:06.426333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6cbbd9-83b3-4878-87a0-00118f19038a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:06.432267 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=79 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:38:06.445124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-ba6cbbd9-83b3-4878-87a0-00118f19038a None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:38:06.446292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-ba6cbbd9-83b3-4878-87a0-00118f19038a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:06.453954 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 742/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:38:06.493909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb24773-b6c3-4b10-a18a-12b2a179ef4f None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:06.499569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=73 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:38:06.506147 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb24773-b6c3-4b10-a18a-12b2a179ef4f None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:06.509839 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278f374-f9b2-4523-b5e2-5a48836988b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:06.510506 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278f374-f9b2-4523-b5e2-5a48836988b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:06.510820 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278f374-f9b2-4523-b5e2-5a48836988b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:06.511137 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278f374-f9b2-4523-b5e2-5a48836988b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:06.520294 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2278f374-f9b2-4523-b5e2-5a48836988b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:06.521717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2278f374-f9b2-4523-b5e2-5a48836988b0 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:06.530919 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 722/1464] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:38:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:38:08.588145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5fe596-a37b-494a-9ef8-c7e637ee15f3 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:08.588712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5fe596-a37b-494a-9ef8-c7e637ee15f3 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:08.588827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5fe596-a37b-494a-9ef8-c7e637ee15f3 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:08.590159 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 743/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:38:08.595344 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647d1f0-519b-402c-be37-4a6ea6a48604 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:08.595763 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647d1f0-519b-402c-be37-4a6ea6a48604 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:08.595927 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647d1f0-519b-402c-be37-4a6ea6a48604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:08.613695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a647d1f0-519b-402c-be37-4a6ea6a48604 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:38:08.614909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a647d1f0-519b-402c-be37-4a6ea6a48604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:08.622915 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 723/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:38:08.658832 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5fe596-a37b-494a-9ef8-c7e637ee15f3 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:08.670181 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5fe596-a37b-494a-9ef8-c7e637ee15f3 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:08.673676 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1d26f-19b7-4bc3-b668-8aacee8de00f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:08.674307 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1d26f-19b7-4bc3-b668-8aacee8de00f service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:08.674628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1d26f-19b7-4bc3-b668-8aacee8de00f service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:08.674906 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1d26f-19b7-4bc3-b668-8aacee8de00f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:08.683950 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda1d26f-19b7-4bc3-b668-8aacee8de00f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:08.685384 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda1d26f-19b7-4bc3-b668-8aacee8de00f service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:08.694870 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 744/1467] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:38:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:38:10.736627 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d6e52-b82e-451d-99da-e8982fa7014a None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:10.737198 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d6e52-b82e-451d-99da-e8982fa7014a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:10.737198 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d6e52-b82e-451d-99da-e8982fa7014a None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:10.739120 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 724/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:38:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:38:10.744395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3d7f8-f831-495a-90a3-5434e21f6f1d None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:10.744828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3d7f8-f831-495a-90a3-5434e21f6f1d None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:10.744926 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3d7f8-f831-495a-90a3-5434e21f6f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:10.762655 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-a4e3d7f8-f831-495a-90a3-5434e21f6f1d None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:38:10.763849 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a4e3d7f8-f831-495a-90a3-5434e21f6f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:10.772114 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 745/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:38:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:38:10.808674 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d6e52-b82e-451d-99da-e8982fa7014a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:10.820029 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d6e52-b82e-451d-99da-e8982fa7014a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:10.823623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59513a26-fc35-4bb0-b893-f9a131fd71fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:10.824237 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59513a26-fc35-4bb0-b893-f9a131fd71fd service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:10.824515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59513a26-fc35-4bb0-b893-f9a131fd71fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:10.824785 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59513a26-fc35-4bb0-b893-f9a131fd71fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:10.834038 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59513a26-fc35-4bb0-b893-f9a131fd71fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:10.835580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59513a26-fc35-4bb0-b893-f9a131fd71fd service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:10.845992 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 725/1470] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:38:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:38:20.757782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=11 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:38:20.841400 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=14 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:38:31.442471 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadf66e-44a8-4479-af8f-be94fc91707e None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:31.443073 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadf66e-44a8-4479-af8f-be94fc91707e None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:31.443073 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadf66e-44a8-4479-af8f-be94fc91707e None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:31.444432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 746/1471] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:38:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:38:31.449867 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d7917-bfac-4a85-a41c-905adc3794fb None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:31.450341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d7917-bfac-4a85-a41c-905adc3794fb None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:31.450443 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d7917-bfac-4a85-a41c-905adc3794fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:31.469018 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-2e4d7917-bfac-4a85-a41c-905adc3794fb None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:38:31.470215 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2e4d7917-bfac-4a85-a41c-905adc3794fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:31.477994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 726/1472] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:38:31.525871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d24b2-bd69-4ebe-abec-dcf55e6567d0 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:31.526160 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d24b2-bd69-4ebe-abec-dcf55e6567d0 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:31.526439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d24b2-bd69-4ebe-abec-dcf55e6567d0 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:31.527834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 747/1473] 10.222.0.13 () {54 vars in 962 bytes} [Thu Jun 23 11:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:38:31.532690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ba154-39c6-4eaf-9528-30f5f7212a04 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:31.532981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ba154-39c6-4eaf-9528-30f5f7212a04 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:31.533261 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ba154-39c6-4eaf-9528-30f5f7212a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:31.551022 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-fa6ba154-39c6-4eaf-9528-30f5f7212a04 None None] MFA Rules not processed for user `b6105838cbf1409587ee7d9e3be554f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:38:31.552213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-fa6ba154-39c6-4eaf-9528-30f5f7212a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:31.558580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 727/1474] 10.222.0.13 () {58 vars in 1059 bytes} [Thu Jun 23 11:38:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:38:31.566144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d24b2-bd69-4ebe-abec-dcf55e6567d0 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:31.576478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d24b2-bd69-4ebe-abec-dcf55e6567d0 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:31.579924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1ec88-17f0-43fb-ba32-c8a65e6e270e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6105838cbf1409587ee7d9e3be554f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:31.580536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1ec88-17f0-43fb-ba32-c8a65e6e270e service glance] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:31.580816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1ec88-17f0-43fb-ba32-c8a65e6e270e service glance] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:31.581087 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1ec88-17f0-43fb-ba32-c8a65e6e270e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:31.590633 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1ec88-17f0-43fb-ba32-c8a65e6e270e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:31.592056 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1ec88-17f0-43fb-ba32-c8a65e6e270e service glance] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:31.601203 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 748/1475] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:38:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:38:33.739560 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696175a-f24c-4b3a-bc7a-596e8d8ca4d8 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:33.740101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696175a-f24c-4b3a-bc7a-596e8d8ca4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:33.740101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696175a-f24c-4b3a-bc7a-596e8d8ca4d8 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:33.741425 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 728/1476] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:38:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:38:33.746785 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fad102-f117-4809-8998-e060339b6153 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:33.747206 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fad102-f117-4809-8998-e060339b6153 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:33.747346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fad102-f117-4809-8998-e060339b6153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:33.765133 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-73fad102-f117-4809-8998-e060339b6153 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:38:33.766333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-73fad102-f117-4809-8998-e060339b6153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:33.774346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 749/1477] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:38:33.797905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696175a-f24c-4b3a-bc7a-596e8d8ca4d8 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:33.810971 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696175a-f24c-4b3a-bc7a-596e8d8ca4d8 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:33.815361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f6963-e829-48fe-b3bb-6688a6063bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:33.817383 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f6963-e829-48fe-b3bb-6688a6063bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:33.817383 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f6963-e829-48fe-b3bb-6688a6063bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:33.817383 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f6963-e829-48fe-b3bb-6688a6063bf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:33.817383 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f6963-e829-48fe-b3bb-6688a6063bf3 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:33.819113 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: WARNING py.warnings [None req-023f6963-e829-48fe-b3bb-6688a6063bf3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:38:33.819113 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: warnings.warn(msg) Jun 23 11:38:33.819113 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: Jun 23 11:38:33.820276 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f6963-e829-48fe-b3bb-6688a6063bf3 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:33.839502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 729/1478] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 23 11:38:33 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:38:35.890675 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5af71c-987e-4c7a-ae38-fb0eeb8b15c3 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:35.891281 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5af71c-987e-4c7a-ae38-fb0eeb8b15c3 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:35.891281 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5af71c-987e-4c7a-ae38-fb0eeb8b15c3 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:35.892594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 750/1479] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:38:35.898026 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24c0a1-6bf0-4645-aff2-b90dbe15a78f None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:35.898420 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24c0a1-6bf0-4645-aff2-b90dbe15a78f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:35.898605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24c0a1-6bf0-4645-aff2-b90dbe15a78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:35.916521 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5a24c0a1-6bf0-4645-aff2-b90dbe15a78f None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:38:35.917710 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5a24c0a1-6bf0-4645-aff2-b90dbe15a78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:35.926305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 730/1480] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:38:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:38:35.969540 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5af71c-987e-4c7a-ae38-fb0eeb8b15c3 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:35.983959 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5af71c-987e-4c7a-ae38-fb0eeb8b15c3 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:35.989757 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb17acc-20ce-4fa4-81e9-c98b1db18ca1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:35.990427 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb17acc-20ce-4fa4-81e9-c98b1db18ca1 service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:35.990802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb17acc-20ce-4fa4-81e9-c98b1db18ca1 service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:35.991093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb17acc-20ce-4fa4-81e9-c98b1db18ca1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:35.999253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb17acc-20ce-4fa4-81e9-c98b1db18ca1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:36.000828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb17acc-20ce-4fa4-81e9-c98b1db18ca1 service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:36.010229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 751/1481] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:38:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:38:38.213743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a60884-dba2-4452-80f1-0256c50e12be None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:38.214385 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a60884-dba2-4452-80f1-0256c50e12be None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:38.214498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a60884-dba2-4452-80f1-0256c50e12be None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:38.215892 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 731/1482] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:38:38.221079 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b27de2-1436-4b93-9c43-316b48578ffa None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:38.221521 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b27de2-1436-4b93-9c43-316b48578ffa None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:38.221692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b27de2-1436-4b93-9c43-316b48578ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:38.239551 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-b7b27de2-1436-4b93-9c43-316b48578ffa None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:38:38.240717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b7b27de2-1436-4b93-9c43-316b48578ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:38.248295 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 752/1483] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:38:38.285826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a60884-dba2-4452-80f1-0256c50e12be None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:38.296387 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a60884-dba2-4452-80f1-0256c50e12be None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:38.299942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2731c-68da-4801-90ef-f6b6a28c64d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:38.300643 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2731c-68da-4801-90ef-f6b6a28c64d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:38.300919 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2731c-68da-4801-90ef-f6b6a28c64d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:38.301188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2731c-68da-4801-90ef-f6b6a28c64d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:38.309877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2731c-68da-4801-90ef-f6b6a28c64d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:38.311326 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2731c-68da-4801-90ef-f6b6a28c64d2 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:38.321102 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 732/1484] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:38:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:38:40.394910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4d7cf-03ab-401b-9e60-1b82421bb03d None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:40.395430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4d7cf-03ab-401b-9e60-1b82421bb03d None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:40.395430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4d7cf-03ab-401b-9e60-1b82421bb03d None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:40.396762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 753/1485] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:38:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:38:40.401798 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043d249-360b-4994-8af9-15c7f90dc2f1 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:40.402179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043d249-360b-4994-8af9-15c7f90dc2f1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:40.402351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043d249-360b-4994-8af9-15c7f90dc2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:40.419926 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5043d249-360b-4994-8af9-15c7f90dc2f1 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:38:40.421106 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5043d249-360b-4994-8af9-15c7f90dc2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:40.428795 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 733/1486] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:38:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:38:40.467547 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4d7cf-03ab-401b-9e60-1b82421bb03d None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:40.478218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4d7cf-03ab-401b-9e60-1b82421bb03d None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:40.481794 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342937b3-708f-40da-bbd2-e1c07f249407 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:40.482433 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342937b3-708f-40da-bbd2-e1c07f249407 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:40.482746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342937b3-708f-40da-bbd2-e1c07f249407 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:40.483021 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342937b3-708f-40da-bbd2-e1c07f249407 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:40.492368 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342937b3-708f-40da-bbd2-e1c07f249407 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:40.493831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342937b3-708f-40da-bbd2-e1c07f249407 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:40.503823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 754/1487] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:38:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:38:42.642274 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea41f7e-2284-45e0-b889-d8d81dddcb85 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:42.642828 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea41f7e-2284-45e0-b889-d8d81dddcb85 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:42.642966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea41f7e-2284-45e0-b889-d8d81dddcb85 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:42.644950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 734/1488] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:38:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:38:42.650124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bf463-54d1-4761-a3ad-ccb1c99dd6f1 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:42.650570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bf463-54d1-4761-a3ad-ccb1c99dd6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:42.650711 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bf463-54d1-4761-a3ad-ccb1c99dd6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:42.668685 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-404bf463-54d1-4761-a3ad-ccb1c99dd6f1 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:38:42.669797 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-404bf463-54d1-4761-a3ad-ccb1c99dd6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:42.677591 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 755/1489] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:38:42.718501 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea41f7e-2284-45e0-b889-d8d81dddcb85 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:42.729573 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea41f7e-2284-45e0-b889-d8d81dddcb85 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:42.733098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a123c-49b5-494f-b920-8e7409f8ce33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:42.733748 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a123c-49b5-494f-b920-8e7409f8ce33 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:42.734030 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a123c-49b5-494f-b920-8e7409f8ce33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:42.734294 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a123c-49b5-494f-b920-8e7409f8ce33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:42.743152 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a123c-49b5-494f-b920-8e7409f8ce33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:42.744638 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a123c-49b5-494f-b920-8e7409f8ce33 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:42.754573 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 735/1490] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:38:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:38:45.150057 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a8fe3-680f-477a-a31f-a529b5ea7f3b None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:45.150662 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a8fe3-680f-477a-a31f-a529b5ea7f3b None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:45.150662 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a8fe3-680f-477a-a31f-a529b5ea7f3b None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:45.152022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 756/1491] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 23 11:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:38:45.157367 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad60e36-87ab-4dd4-b623-968e84939bb5 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:45.157789 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad60e36-87ab-4dd4-b623-968e84939bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:45.157924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad60e36-87ab-4dd4-b623-968e84939bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:45.175365 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-dad60e36-87ab-4dd4-b623-968e84939bb5 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:38:45.176548 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-dad60e36-87ab-4dd4-b623-968e84939bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:45.184448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 736/1492] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 23 11:38:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:38:45.222941 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a8fe3-680f-477a-a31f-a529b5ea7f3b None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:45.233666 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a8fe3-680f-477a-a31f-a529b5ea7f3b None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:45.237687 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ffe5b-7f44-4f02-bd85-3de7f2cf8fbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:45.238320 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ffe5b-7f44-4f02-bd85-3de7f2cf8fbc service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:45.238641 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ffe5b-7f44-4f02-bd85-3de7f2cf8fbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:45.238919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ffe5b-7f44-4f02-bd85-3de7f2cf8fbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:45.247740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ffe5b-7f44-4f02-bd85-3de7f2cf8fbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:45.249206 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ffe5b-7f44-4f02-bd85-3de7f2cf8fbc service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:45.265655 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 757/1493] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:38:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:38:55.170211 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=33 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:38:55.259139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=33 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:24.799813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acec5f-bc0f-46ba-ba14-99228d3a5595 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:24.800370 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acec5f-bc0f-46ba-ba14-99228d3a5595 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:24.800370 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acec5f-bc0f-46ba-ba14-99228d3a5595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:24.817950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-10acec5f-bc0f-46ba-ba14-99228d3a5595 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:40:24.819154 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-10acec5f-bc0f-46ba-ba14-99228d3a5595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:24.826809 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 737/1494] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:40:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:40:24.836697 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4468aa7-d26c-493b-9276-8dfb9131fbcd None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:24.837218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4468aa7-d26c-493b-9276-8dfb9131fbcd None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:24.837322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4468aa7-d26c-493b-9276-8dfb9131fbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:24.855246 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-b4468aa7-d26c-493b-9276-8dfb9131fbcd None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:40:24.856423 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b4468aa7-d26c-493b-9276-8dfb9131fbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:24.864137 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 758/1495] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:40:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:40:24.868660 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acec5f-bc0f-46ba-ba14-99228d3a5595 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:24.878220 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acec5f-bc0f-46ba-ba14-99228d3a5595 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:24.883292 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab22fa-a9ea-4731-9208-131ac1eca364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:24.883940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab22fa-a9ea-4731-9208-131ac1eca364 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:24.884222 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab22fa-a9ea-4731-9208-131ac1eca364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:24.884498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab22fa-a9ea-4731-9208-131ac1eca364 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:24.885219 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab22fa-a9ea-4731-9208-131ac1eca364 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:40:24.885525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab22fa-a9ea-4731-9208-131ac1eca364 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:24.886539 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: WARNING py.warnings [None req-4aab22fa-a9ea-4731-9208-131ac1eca364 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:40:24.886539 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: warnings.warn(msg) Jun 23 11:40:24.886539 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: Jun 23 11:40:24.887467 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab22fa-a9ea-4731-9208-131ac1eca364 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:24.899407 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 738/1496] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:40:24.907200 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4468aa7-d26c-493b-9276-8dfb9131fbcd None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:24.916311 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4468aa7-d26c-493b-9276-8dfb9131fbcd None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:24.920820 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac517846-c31d-4e7a-9cb0-f27b344e89e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:24.921455 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac517846-c31d-4e7a-9cb0-f27b344e89e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:24.921733 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac517846-c31d-4e7a-9cb0-f27b344e89e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:24.922004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac517846-c31d-4e7a-9cb0-f27b344e89e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:24.922727 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac517846-c31d-4e7a-9cb0-f27b344e89e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:24.924463 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: WARNING py.warnings [None req-ac517846-c31d-4e7a-9cb0-f27b344e89e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:40:24.924463 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: warnings.warn(msg) Jun 23 11:40:24.924463 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: Jun 23 11:40:24.925653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac517846-c31d-4e7a-9cb0-f27b344e89e2 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:25.043091 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 759/1497] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:40:24 2022] POST /identity/v3/projects => generated 377 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:40:25.047258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab22fa-a9ea-4731-9208-131ac1eca364 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:25.057313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab22fa-a9ea-4731-9208-131ac1eca364 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:25.113461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f3642-a7c8-4bd8-b358-a84bd7060829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:25.114124 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f3642-a7c8-4bd8-b358-a84bd7060829 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:25.114407 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f3642-a7c8-4bd8-b358-a84bd7060829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:25.114715 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f3642-a7c8-4bd8-b358-a84bd7060829 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:25.115399 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f3642-a7c8-4bd8-b358-a84bd7060829 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:25.116405 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: WARNING py.warnings [None req-c82f3642-a7c8-4bd8-b358-a84bd7060829 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:40:25.116405 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: warnings.warn(msg) Jun 23 11:40:25.116405 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: Jun 23 11:40:25.117620 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f3642-a7c8-4bd8-b358-a84bd7060829 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:25.229778 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 739/1498] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:40:25 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:40:25.233875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac517846-c31d-4e7a-9cb0-f27b344e89e2 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:25.244123 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac517846-c31d-4e7a-9cb0-f27b344e89e2 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:25.248552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d23a1f-39f9-4419-9a3b-1fa53f174461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:25.249202 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d23a1f-39f9-4419-9a3b-1fa53f174461 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:25.249484 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d23a1f-39f9-4419-9a3b-1fa53f174461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:25.249760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d23a1f-39f9-4419-9a3b-1fa53f174461 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:25.250477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d23a1f-39f9-4419-9a3b-1fa53f174461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:25.251508 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: WARNING py.warnings [None req-17d23a1f-39f9-4419-9a3b-1fa53f174461 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:40:25.251508 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: warnings.warn(msg) Jun 23 11:40:25.251508 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: Jun 23 11:40:25.252388 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d23a1f-39f9-4419-9a3b-1fa53f174461 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:25.265607 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 760/1499] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:40:25.270423 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f3642-a7c8-4bd8-b358-a84bd7060829 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:25.280301 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f3642-a7c8-4bd8-b358-a84bd7060829 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:25.284828 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569d482-5067-41d7-ad66-7c5f179829b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:25.285545 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569d482-5067-41d7-ad66-7c5f179829b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:25.285826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569d482-5067-41d7-ad66-7c5f179829b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:25.286109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569d482-5067-41d7-ad66-7c5f179829b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:25.286880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569d482-5067-41d7-ad66-7c5f179829b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:25.287878 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: WARNING py.warnings [None req-0569d482-5067-41d7-ad66-7c5f179829b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:40:25.287878 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: warnings.warn(msg) Jun 23 11:40:25.287878 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: Jun 23 11:40:25.288754 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569d482-5067-41d7-ad66-7c5f179829b0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:25.302740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 740/1500] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:40:25.307035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d23a1f-39f9-4419-9a3b-1fa53f174461 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:25.317636 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d23a1f-39f9-4419-9a3b-1fa53f174461 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:25.322106 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fdcfde-fca4-4c48-a22f-06befaec3bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:25.323006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdcfde-fca4-4c48-a22f-06befaec3bf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:25.323295 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdcfde-fca4-4c48-a22f-06befaec3bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:25.323573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdcfde-fca4-4c48-a22f-06befaec3bf9 admin admin] PATH_INFO: `/v3/projects/13003690ab8648f48418ff511c936a72/users/8461a6344eb444a6a27c391e936eab13/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:25.338329 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fdcfde-fca4-4c48-a22f-06befaec3bf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13003690ab8648f48418ff511c936a72, user_id=8461a6344eb444a6a27c391e936eab13, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:25.339396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: WARNING py.warnings [None req-a9fdcfde-fca4-4c48-a22f-06befaec3bf9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:40:25.339396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: warnings.warn(msg) Jun 23 11:40:25.339396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: Jun 23 11:40:25.341336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fdcfde-fca4-4c48-a22f-06befaec3bf9 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:25.441399 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 761/1501] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:40:25 2022] PUT /identity/v3/projects/13003690ab8648f48418ff511c936a72/users/8461a6344eb444a6a27c391e936eab13/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:40:25.453204 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0793e-2f4b-4cc3-ab3d-efc1491fb418 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:25.453498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0793e-2f4b-4cc3-ab3d-efc1491fb418 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:25.453775 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0793e-2f4b-4cc3-ab3d-efc1491fb418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:25.480130 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5ee0793e-2f4b-4cc3-ab3d-efc1491fb418 None None] MFA Rules not processed for user `8461a6344eb444a6a27c391e936eab13`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:40:25.481323 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5ee0793e-2f4b-4cc3-ab3d-efc1491fb418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:25.546255 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 741/1502] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:40:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:40:25.560047 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d042c15-b0c6-47e4-9ddf-330531c60d86 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:25.560337 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d042c15-b0c6-47e4-9ddf-330531c60d86 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:25.560610 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d042c15-b0c6-47e4-9ddf-330531c60d86 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:25.561958 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 762/1503] 10.222.0.13 () {54 vars in 963 bytes} [Thu Jun 23 11:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:40:25.567433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a721d-1e1a-4a6d-8c01-0c1f5b3d4a04 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:25.567791 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a721d-1e1a-4a6d-8c01-0c1f5b3d4a04 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:25.568005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a721d-1e1a-4a6d-8c01-0c1f5b3d4a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:25.585748 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ff3a721d-1e1a-4a6d-8c01-0c1f5b3d4a04 None None] MFA Rules not processed for user `871dd684032c49d09608f0af05b7a765`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:40:25.587046 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ff3a721d-1e1a-4a6d-8c01-0c1f5b3d4a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:25.629952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:25.637133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 742/1504] 10.222.0.13 () {58 vars in 1060 bytes} [Thu Jun 23 11:40:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:40:25.644952 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d042c15-b0c6-47e4-9ddf-330531c60d86 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:25.655427 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d042c15-b0c6-47e4-9ddf-330531c60d86 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:25.658959 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441e419-016f-48db-8ad0-c51f04f850c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:25.659602 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441e419-016f-48db-8ad0-c51f04f850c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:25.660413 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441e419-016f-48db-8ad0-c51f04f850c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:25.660752 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441e419-016f-48db-8ad0-c51f04f850c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:25.668943 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4441e419-016f-48db-8ad0-c51f04f850c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:25.670398 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4441e419-016f-48db-8ad0-c51f04f850c6 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:25.679767 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 763/1505] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:40:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:40:25.721192 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da839ce-0f45-4251-9ee4-dcb3217136d2 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:25.721483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da839ce-0f45-4251-9ee4-dcb3217136d2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:25.721759 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da839ce-0f45-4251-9ee4-dcb3217136d2 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:25.723776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 743/1506] 10.222.0.13 () {52 vars in 829 bytes} [Thu Jun 23 11:40:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:40:25.727431 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441e419-016f-48db-8ad0-c51f04f850c6 service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:25.788860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441e419-016f-48db-8ad0-c51f04f850c6 service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:25.793344 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de710f6-0c63-42eb-8659-dc4bb3f47772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:25.794203 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de710f6-0c63-42eb-8659-dc4bb3f47772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:25.794491 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de710f6-0c63-42eb-8659-dc4bb3f47772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:25.794810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de710f6-0c63-42eb-8659-dc4bb3f47772 admin admin] PATH_INFO: `/v3/users/8461a6344eb444a6a27c391e936eab13` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:25.795986 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de710f6-0c63-42eb-8659-dc4bb3f47772 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8461a6344eb444a6a27c391e936eab13)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:25.796994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: WARNING py.warnings [None req-0de710f6-0c63-42eb-8659-dc4bb3f47772 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:40:25.796994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: warnings.warn(msg) Jun 23 11:40:25.796994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: Jun 23 11:40:25.798197 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de710f6-0c63-42eb-8659-dc4bb3f47772 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:25.984414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-0de710f6-0c63-42eb-8659-dc4bb3f47772 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8461a6344eb444a6a27c391e936eab13'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:40:25.996585 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-0de710f6-0c63-42eb-8659-dc4bb3f47772 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8461a6344eb444a6a27c391e936eab13'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:40:25.997403 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-0de710f6-0c63-42eb-8659-dc4bb3f47772 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8461a6344eb444a6a27c391e936eab13'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:40:25.998042 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-0de710f6-0c63-42eb-8659-dc4bb3f47772 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8461a6344eb444a6a27c391e936eab13'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:40:26.016445 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-0de710f6-0c63-42eb-8659-dc4bb3f47772 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8461a6344eb444a6a27c391e936eab13'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:40:26.021052 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:26.109659 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 764/1507] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:40:25 2022] DELETE /identity/v3/users/8461a6344eb444a6a27c391e936eab13 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:40:26.118734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68ac2d-25e1-47b5-90a3-51eb3ad3c9f3 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:26.119210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68ac2d-25e1-47b5-90a3-51eb3ad3c9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:26.119303 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68ac2d-25e1-47b5-90a3-51eb3ad3c9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:26.137206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3a68ac2d-25e1-47b5-90a3-51eb3ad3c9f3 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:40:26.138401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3a68ac2d-25e1-47b5-90a3-51eb3ad3c9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:26.196384 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 744/1508] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:40:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:40:26.206067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de710f6-0c63-42eb-8659-dc4bb3f47772 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:26.207529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-0de710f6-0c63-42eb-8659-dc4bb3f47772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:26.266443 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de710f6-0c63-42eb-8659-dc4bb3f47772 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:26.270386 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb4316-ef40-4f25-8b17-7bd8baf95082 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:26.271059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb4316-ef40-4f25-8b17-7bd8baf95082 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:26.271344 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb4316-ef40-4f25-8b17-7bd8baf95082 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:26.271619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb4316-ef40-4f25-8b17-7bd8baf95082 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:26.280384 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb4316-ef40-4f25-8b17-7bd8baf95082 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:26.281829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb4316-ef40-4f25-8b17-7bd8baf95082 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:26.291312 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 765/1509] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:40:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:40:26.757985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68ac2d-25e1-47b5-90a3-51eb3ad3c9f3 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:26.759672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3a68ac2d-25e1-47b5-90a3-51eb3ad3c9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:26.772620 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68ac2d-25e1-47b5-90a3-51eb3ad3c9f3 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:26.777328 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80637421-6220-47e2-bd3e-07608c2b3e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:26.778489 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80637421-6220-47e2-bd3e-07608c2b3e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:26.778868 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80637421-6220-47e2-bd3e-07608c2b3e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:26.779205 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80637421-6220-47e2-bd3e-07608c2b3e4a admin admin] PATH_INFO: `/v3/projects/13003690ab8648f48418ff511c936a72` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:26.780496 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80637421-6220-47e2-bd3e-07608c2b3e4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=13003690ab8648f48418ff511c936a72)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:26.781701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: WARNING py.warnings [None req-80637421-6220-47e2-bd3e-07608c2b3e4a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:40:26.781701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: warnings.warn(msg) Jun 23 11:40:26.781701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: Jun 23 11:40:26.785869 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80637421-6220-47e2-bd3e-07608c2b3e4a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:26.880964 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-80637421-6220-47e2-bd3e-07608c2b3e4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13003690ab8648f48418ff511c936a72', 'request_id': 'req-80637421-6220-47e2-bd3e-07608c2b3e4a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:40:26.894007 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-80637421-6220-47e2-bd3e-07608c2b3e4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13003690ab8648f48418ff511c936a72', 'request_id': 'req-80637421-6220-47e2-bd3e-07608c2b3e4a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:40:26.954773 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-80637421-6220-47e2-bd3e-07608c2b3e4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13003690ab8648f48418ff511c936a72', 'request_id': 'req-80637421-6220-47e2-bd3e-07608c2b3e4a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:40:26.966996 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-80637421-6220-47e2-bd3e-07608c2b3e4a admin admin] The token cache is being invalidate because project 13003690ab8648f48418ff511c936a72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:40:26.967290 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-80637421-6220-47e2-bd3e-07608c2b3e4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:40:26.968047 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-80637421-6220-47e2-bd3e-07608c2b3e4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:40:26.969721 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 745/1510] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:40:26 2022] DELETE /identity/v3/projects/13003690ab8648f48418ff511c936a72 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:40:36.287372 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:36.963759 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:03.882784 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951d976-a269-4a09-bf0d-ef67e26f53e2 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:03.883371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951d976-a269-4a09-bf0d-ef67e26f53e2 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:03.883371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951d976-a269-4a09-bf0d-ef67e26f53e2 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:03.884756 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 766/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:03.889893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1228faa-f5b2-46cd-acee-45e9e878623f None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:03.890430 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1228faa-f5b2-46cd-acee-45e9e878623f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:03.890579 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1228faa-f5b2-46cd-acee-45e9e878623f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:03.909092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-e1228faa-f5b2-46cd-acee-45e9e878623f None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:03.911095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e1228faa-f5b2-46cd-acee-45e9e878623f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:03.971758 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 746/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:42:04.137790 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951d976-a269-4a09-bf0d-ef67e26f53e2 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:04.139466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-1951d976-a269-4a09-bf0d-ef67e26f53e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:04.214031 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951d976-a269-4a09-bf0d-ef67e26f53e2 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:04.219275 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b33f0-c9ec-455a-87e4-adf40460951f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:04.219914 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b33f0-c9ec-455a-87e4-adf40460951f service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:04.220194 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b33f0-c9ec-455a-87e4-adf40460951f service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:04.220469 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b33f0-c9ec-455a-87e4-adf40460951f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:04.229361 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b33f0-c9ec-455a-87e4-adf40460951f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:04.230994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b33f0-c9ec-455a-87e4-adf40460951f service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:04.240525 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 767/1513] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:42:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:42:05.075419 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea68a8-6eb6-4495-955d-2085d8f3e85e None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:05.075991 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea68a8-6eb6-4495-955d-2085d8f3e85e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:05.075991 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea68a8-6eb6-4495-955d-2085d8f3e85e None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:05.077575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 747/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:05.083314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5efde1-d2eb-4cc6-9024-9a412598ef97 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:05.094263 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5efde1-d2eb-4cc6-9024-9a412598ef97 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:05.094263 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5efde1-d2eb-4cc6-9024-9a412598ef97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:05.105142 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-9d5efde1-d2eb-4cc6-9024-9a412598ef97 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:05.106354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-9d5efde1-d2eb-4cc6-9024-9a412598ef97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:05.115425 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 768/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:42:05.266653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea68a8-6eb6-4495-955d-2085d8f3e85e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:05.280773 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea68a8-6eb6-4495-955d-2085d8f3e85e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:05.286688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59819d-078a-44a4-8195-1c2a7c322276 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:05.287323 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59819d-078a-44a4-8195-1c2a7c322276 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:05.287607 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59819d-078a-44a4-8195-1c2a7c322276 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:05.287883 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59819d-078a-44a4-8195-1c2a7c322276 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:05.295785 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc59819d-078a-44a4-8195-1c2a7c322276 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:05.297377 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc59819d-078a-44a4-8195-1c2a7c322276 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:05.308317 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 748/1516] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:42:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:42:06.687019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6db354-3cdd-4b95-aece-58ffb2e2539f None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:06.687558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6db354-3cdd-4b95-aece-58ffb2e2539f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:06.687558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6db354-3cdd-4b95-aece-58ffb2e2539f None None] PATH_INFO: `/v3` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:06.688889 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 769/1517] 10.222.0.46 () {54 vars in 925 bytes} [Thu Jun 23 11:42:06 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:06.695368 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8b9a0-bc7e-4a11-b0a3-1d47f40d14f7 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:06.695807 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8b9a0-bc7e-4a11-b0a3-1d47f40d14f7 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:06.695925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8b9a0-bc7e-4a11-b0a3-1d47f40d14f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:06.713955 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c2a8b9a0-bc7e-4a11-b0a3-1d47f40d14f7 None None] MFA Rules not processed for user `ac04b32777ca4f929e9e59deee6e0260`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:06.715197 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c2a8b9a0-bc7e-4a11-b0a3-1d47f40d14f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:06.777184 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 749/1518] 10.222.0.46 () {58 vars in 1015 bytes} [Thu Jun 23 11:42:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:42:10.803038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809c478-8075-4027-940e-73f7bdaaef4c None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:10.803588 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809c478-8075-4027-940e-73f7bdaaef4c None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:10.803684 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809c478-8075-4027-940e-73f7bdaaef4c None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:10.805059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 770/1519] 10.222.0.13 () {54 vars in 960 bytes} [Thu Jun 23 11:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:10.810251 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eba56b-68b1-42e3-b8e0-5c0dbfd73050 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:10.810700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eba56b-68b1-42e3-b8e0-5c0dbfd73050 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:10.810839 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eba56b-68b1-42e3-b8e0-5c0dbfd73050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:10.829252 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d5eba56b-68b1-42e3-b8e0-5c0dbfd73050 None None] MFA Rules not processed for user `ac04b32777ca4f929e9e59deee6e0260`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:10.830455 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d5eba56b-68b1-42e3-b8e0-5c0dbfd73050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:10.839885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 750/1520] 10.222.0.13 () {58 vars in 1057 bytes} [Thu Jun 23 11:42:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:42:10.847067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0809c478-8075-4027-940e-73f7bdaaef4c None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:10.860071 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0809c478-8075-4027-940e-73f7bdaaef4c None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:10.864817 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f271df8-6ace-417a-8f30-5fac8f14ff5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac04b32777ca4f929e9e59deee6e0260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:10.865572 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f271df8-6ace-417a-8f30-5fac8f14ff5b service placement] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:10.865852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f271df8-6ace-417a-8f30-5fac8f14ff5b service placement] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:10.866125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f271df8-6ace-417a-8f30-5fac8f14ff5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:10.874134 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f271df8-6ace-417a-8f30-5fac8f14ff5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:10.875740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f271df8-6ace-417a-8f30-5fac8f14ff5b service placement] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:10.885571 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 771/1521] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:42:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:42:16.534205 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a2460-6f14-4ba2-9d8a-0ae2a28584fd None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:16.534761 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a2460-6f14-4ba2-9d8a-0ae2a28584fd None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:16.534865 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a2460-6f14-4ba2-9d8a-0ae2a28584fd None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:16.536172 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 751/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:16.541550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c6c06-957c-4338-8e00-0afb7e9e8444 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:16.541998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c6c06-957c-4338-8e00-0afb7e9e8444 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:16.542093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c6c06-957c-4338-8e00-0afb7e9e8444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:16.681555 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-1f9c6c06-957c-4338-8e00-0afb7e9e8444 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:16.682794 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-1f9c6c06-957c-4338-8e00-0afb7e9e8444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:16.690496 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 772/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:42:16.849355 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a2460-6f14-4ba2-9d8a-0ae2a28584fd None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:16.863530 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a2460-6f14-4ba2-9d8a-0ae2a28584fd None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:16.869098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddff24b-9b5a-4959-a9f5-e7c83058324d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:16.869732 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddff24b-9b5a-4959-a9f5-e7c83058324d service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:16.870012 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddff24b-9b5a-4959-a9f5-e7c83058324d service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:16.870288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddff24b-9b5a-4959-a9f5-e7c83058324d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:16.878613 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddff24b-9b5a-4959-a9f5-e7c83058324d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:16.880204 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddff24b-9b5a-4959-a9f5-e7c83058324d service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:16.890241 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 752/1524] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:42:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:42:17.706975 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39841cf1-519b-4d74-aa8e-416d668597a0 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:17.707555 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39841cf1-519b-4d74-aa8e-416d668597a0 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:17.707555 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39841cf1-519b-4d74-aa8e-416d668597a0 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:17.709221 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 773/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:17.713966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b77e02-a0ee-4b45-bd51-06b4b85e04a4 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:17.714416 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b77e02-a0ee-4b45-bd51-06b4b85e04a4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:17.714525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b77e02-a0ee-4b45-bd51-06b4b85e04a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:17.732142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-50b77e02-a0ee-4b45-bd51-06b4b85e04a4 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:17.733326 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-50b77e02-a0ee-4b45-bd51-06b4b85e04a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:17.741569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 753/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:42:17.894656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39841cf1-519b-4d74-aa8e-416d668597a0 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:17.908902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39841cf1-519b-4d74-aa8e-416d668597a0 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:17.914296 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b43bc-48b8-48d4-9901-5a994f2826ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:17.914954 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b43bc-48b8-48d4-9901-5a994f2826ef service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:17.915238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b43bc-48b8-48d4-9901-5a994f2826ef service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:17.915510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b43bc-48b8-48d4-9901-5a994f2826ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:17.923432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b43bc-48b8-48d4-9901-5a994f2826ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:17.925029 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b43bc-48b8-48d4-9901-5a994f2826ef service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:17.934741 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 774/1527] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:42:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:42:22.104245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7725e0-b18d-42fe-b48e-5c82f5aeeca0 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:22.104825 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7725e0-b18d-42fe-b48e-5c82f5aeeca0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:22.104825 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7725e0-b18d-42fe-b48e-5c82f5aeeca0 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:22.106259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 754/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:22.111592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94720a8d-65b4-4ecb-bd7d-ddedc618dada None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:22.112008 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94720a8d-65b4-4ecb-bd7d-ddedc618dada None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:22.112143 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94720a8d-65b4-4ecb-bd7d-ddedc618dada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:22.130069 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-94720a8d-65b4-4ecb-bd7d-ddedc618dada None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:22.131329 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-94720a8d-65b4-4ecb-bd7d-ddedc618dada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:22.139014 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 775/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:42:22.176656 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7725e0-b18d-42fe-b48e-5c82f5aeeca0 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:22.179169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3c7725e0-b18d-42fe-b48e-5c82f5aeeca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:22.237277 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7725e0-b18d-42fe-b48e-5c82f5aeeca0 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:22.240936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62cbee-04d2-4d14-8a14-c78bb88b0e3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:22.241581 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62cbee-04d2-4d14-8a14-c78bb88b0e3d service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:22.241869 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62cbee-04d2-4d14-8a14-c78bb88b0e3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:22.242149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62cbee-04d2-4d14-8a14-c78bb88b0e3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:22.251043 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62cbee-04d2-4d14-8a14-c78bb88b0e3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:22.252471 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62cbee-04d2-4d14-8a14-c78bb88b0e3d service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:22.257369 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:22.262196 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 755/1530] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:42:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:42:24.300651 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb5f2b-468f-479f-a50d-d17649e00fb9 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:24.301186 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb5f2b-468f-479f-a50d-d17649e00fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:24.301186 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb5f2b-468f-479f-a50d-d17649e00fb9 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:24.303222 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 776/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:42:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:24.308482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d504d-303a-458a-83fb-6a60228fb563 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:24.308917 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d504d-303a-458a-83fb-6a60228fb563 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:24.309028 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d504d-303a-458a-83fb-6a60228fb563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:24.326566 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-e25d504d-303a-458a-83fb-6a60228fb563 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:24.327739 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e25d504d-303a-458a-83fb-6a60228fb563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:24.335664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 756/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:42:24.372009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb5f2b-468f-479f-a50d-d17649e00fb9 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:24.383434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb5f2b-468f-479f-a50d-d17649e00fb9 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:24.386971 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2d595-a71c-4f34-b194-372957da2a2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:24.387702 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2d595-a71c-4f34-b194-372957da2a2d service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:24.387896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2d595-a71c-4f34-b194-372957da2a2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:24.388171 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2d595-a71c-4f34-b194-372957da2a2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:24.397500 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee2d595-a71c-4f34-b194-372957da2a2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:24.398969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee2d595-a71c-4f34-b194-372957da2a2d service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:24.408978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 777/1533] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:42:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:42:28.031476 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a48a1c-9f66-4c2f-ab70-b4ca742cef57 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:28.032080 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a48a1c-9f66-4c2f-ab70-b4ca742cef57 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:28.032080 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a48a1c-9f66-4c2f-ab70-b4ca742cef57 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:28.033432 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 757/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:28.039092 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26a9c3-18b7-4efb-8595-ff9cc007681c None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:28.039553 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26a9c3-18b7-4efb-8595-ff9cc007681c None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:28.039706 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26a9c3-18b7-4efb-8595-ff9cc007681c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:28.045749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=62 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:28.057949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-fa26a9c3-18b7-4efb-8595-ff9cc007681c None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:28.059274 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-fa26a9c3-18b7-4efb-8595-ff9cc007681c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:28.067224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 778/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:42:28.370135 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a48a1c-9f66-4c2f-ab70-b4ca742cef57 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:28.371971 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-17a48a1c-9f66-4c2f-ab70-b4ca742cef57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:28.430239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a48a1c-9f66-4c2f-ab70-b4ca742cef57 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:28.433864 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef7feb-35d0-4567-8dcd-46500d858a08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:28.434520 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef7feb-35d0-4567-8dcd-46500d858a08 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:28.434844 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef7feb-35d0-4567-8dcd-46500d858a08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:28.435125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef7feb-35d0-4567-8dcd-46500d858a08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:28.444333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef7feb-35d0-4567-8dcd-46500d858a08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:28.445776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef7feb-35d0-4567-8dcd-46500d858a08 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:28.455223 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 758/1536] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:42:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:42:30.484085 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad2e2c-81f7-4b46-806a-ce06175eb52f None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:30.484602 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad2e2c-81f7-4b46-806a-ce06175eb52f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:30.484602 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad2e2c-81f7-4b46-806a-ce06175eb52f None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:30.485920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 779/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:30.491375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86282a72-6f74-424f-80b7-71ededf25f16 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:30.491810 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86282a72-6f74-424f-80b7-71ededf25f16 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:30.491963 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86282a72-6f74-424f-80b7-71ededf25f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:30.509926 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-86282a72-6f74-424f-80b7-71ededf25f16 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:30.511149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-86282a72-6f74-424f-80b7-71ededf25f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:30.519024 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 759/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:42:30.821151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad2e2c-81f7-4b46-806a-ce06175eb52f None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:30.822864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d7ad2e2c-81f7-4b46-806a-ce06175eb52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:30.835862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad2e2c-81f7-4b46-806a-ce06175eb52f None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:30.839351 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59324504-aa66-430b-ad88-094ee3947a64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:30.839978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59324504-aa66-430b-ad88-094ee3947a64 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:30.840259 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59324504-aa66-430b-ad88-094ee3947a64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:30.840532 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59324504-aa66-430b-ad88-094ee3947a64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:30.849737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59324504-aa66-430b-ad88-094ee3947a64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:30.851199 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59324504-aa66-430b-ad88-094ee3947a64 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:30.860448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 780/1539] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:42:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:42:40.504739 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=32 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:40.856306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=8 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:58.557082 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246960e-dbb6-4652-a16f-c03b0edd5c7e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:58.558989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246960e-dbb6-4652-a16f-c03b0edd5c7e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:58.558989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246960e-dbb6-4652-a16f-c03b0edd5c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:58.583275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a246960e-dbb6-4652-a16f-c03b0edd5c7e None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:58.585161 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a246960e-dbb6-4652-a16f-c03b0edd5c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:58.597521 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 760/1540] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:58.624179 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfbf72-c3cb-413e-b4ba-3278cbd8be70 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:58.625357 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfbf72-c3cb-413e-b4ba-3278cbd8be70 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:58.625865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfbf72-c3cb-413e-b4ba-3278cbd8be70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:58.647096 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff51c88-f4c2-4e3c-8650-f3d098c4c8f3 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:58.647780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff51c88-f4c2-4e3c-8650-f3d098c4c8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:58.648291 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff51c88-f4c2-4e3c-8650-f3d098c4c8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:58.679591 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-96bfbf72-c3cb-413e-b4ba-3278cbd8be70 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:58.681248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-96bfbf72-c3cb-413e-b4ba-3278cbd8be70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:58.696309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-4ff51c88-f4c2-4e3c-8650-f3d098c4c8f3 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:58.702348 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 781/1541] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:58.703307 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4ff51c88-f4c2-4e3c-8650-f3d098c4c8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:58.707603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bfbf72-c3cb-413e-b4ba-3278cbd8be70 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:58.720872 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 761/1542] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:43:58.729579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bfbf72-c3cb-413e-b4ba-3278cbd8be70 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:58.731581 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809833a2-b355-4c7d-9326-8164dcc1b67b None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:58.732211 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809833a2-b355-4c7d-9326-8164dcc1b67b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:58.732861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809833a2-b355-4c7d-9326-8164dcc1b67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:58.747727 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8084c3-6be6-4179-9954-d73fa54817c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:58.748938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8084c3-6be6-4179-9954-d73fa54817c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:58.751682 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8084c3-6be6-4179-9954-d73fa54817c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:58.752349 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8084c3-6be6-4179-9954-d73fa54817c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:58.753390 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8084c3-6be6-4179-9954-d73fa54817c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:58.757997 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8084c3-6be6-4179-9954-d73fa54817c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:58.759457 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: WARNING py.warnings [None req-4a8084c3-6be6-4179-9954-d73fa54817c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:43:58.759457 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: warnings.warn(msg) Jun 23 11:43:58.759457 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: Jun 23 11:43:58.760797 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8084c3-6be6-4179-9954-d73fa54817c3 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:58.779266 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-809833a2-b355-4c7d-9326-8164dcc1b67b None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:58.781612 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-809833a2-b355-4c7d-9326-8164dcc1b67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:58.790185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 782/1543] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:58.792101 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017262f-096f-49d7-9c39-ed1fa2e94de9 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:58.792721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017262f-096f-49d7-9c39-ed1fa2e94de9 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:58.793243 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017262f-096f-49d7-9c39-ed1fa2e94de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:58.808160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 762/1544] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:58.809486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809833a2-b355-4c7d-9326-8164dcc1b67b None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:58.815009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-8017262f-096f-49d7-9c39-ed1fa2e94de9 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:58.816712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-8017262f-096f-49d7-9c39-ed1fa2e94de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:58.829485 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809833a2-b355-4c7d-9326-8164dcc1b67b None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:58.832194 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 783/1545] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:58.833371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017262f-096f-49d7-9c39-ed1fa2e94de9 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:58.849466 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a636d7-7bd3-40ab-b5ee-de4fb7f5fa4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:58.849743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a636d7-7bd3-40ab-b5ee-de4fb7f5fa4d admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:58.849851 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a636d7-7bd3-40ab-b5ee-de4fb7f5fa4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:58.850089 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a636d7-7bd3-40ab-b5ee-de4fb7f5fa4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:58.850843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a636d7-7bd3-40ab-b5ee-de4fb7f5fa4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:58.851851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017262f-096f-49d7-9c39-ed1fa2e94de9 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:58.851991 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: WARNING py.warnings [None req-88a636d7-7bd3-40ab-b5ee-de4fb7f5fa4d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:43:58.851991 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: warnings.warn(msg) Jun 23 11:43:58.851991 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: Jun 23 11:43:58.853138 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a636d7-7bd3-40ab-b5ee-de4fb7f5fa4d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:58.857594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02bc371-2e7d-45c0-bc93-38c963e3476f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:58.859415 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bc371-2e7d-45c0-bc93-38c963e3476f admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:58.859950 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bc371-2e7d-45c0-bc93-38c963e3476f admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:58.860451 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bc371-2e7d-45c0-bc93-38c963e3476f admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:58.861710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02bc371-2e7d-45c0-bc93-38c963e3476f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:58.866555 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02bc371-2e7d-45c0-bc93-38c963e3476f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:58.868747 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02bc371-2e7d-45c0-bc93-38c963e3476f admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:58.883554 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 784/1546] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:58.885439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e13aa-715e-4e87-a4d0-3ffa6be619cb None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:58.885965 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e13aa-715e-4e87-a4d0-3ffa6be619cb None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:58.886466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e13aa-715e-4e87-a4d0-3ffa6be619cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:58.907687 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-5d5e13aa-715e-4e87-a4d0-3ffa6be619cb None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:58.918690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-5d5e13aa-715e-4e87-a4d0-3ffa6be619cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:58.923466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 785/1547] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:58.924841 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ebe25-83ed-4018-96c7-95a3a6441bd5 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:58.925130 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ebe25-83ed-4018-96c7-95a3a6441bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:58.925406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ebe25-83ed-4018-96c7-95a3a6441bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:58.949926 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c43ebe25-83ed-4018-96c7-95a3a6441bd5 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:58.951664 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c43ebe25-83ed-4018-96c7-95a3a6441bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:58.964482 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 786/1548] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:58.965405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ebe25-83ed-4018-96c7-95a3a6441bd5 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:58.976414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ebe25-83ed-4018-96c7-95a3a6441bd5 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:58.980536 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 763/1549] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:43:58 2022] POST /identity/v3/projects => generated 371 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:58.981546 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a636d7-7bd3-40ab-b5ee-de4fb7f5fa4d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:58.981751 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27632eee-f5cc-4a04-90fc-6a9fe6dbfa38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:58.982389 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27632eee-f5cc-4a04-90fc-6a9fe6dbfa38 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:58.982706 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27632eee-f5cc-4a04-90fc-6a9fe6dbfa38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:58.983548 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27632eee-f5cc-4a04-90fc-6a9fe6dbfa38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:58.984250 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27632eee-f5cc-4a04-90fc-6a9fe6dbfa38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:58.986651 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27632eee-f5cc-4a04-90fc-6a9fe6dbfa38 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:59.057459 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a636d7-7bd3-40ab-b5ee-de4fb7f5fa4d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:59.062262 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4e626-83ea-469e-a54d-c425036c0b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:59.062982 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4e626-83ea-469e-a54d-c425036c0b9b admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.063273 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4e626-83ea-469e-a54d-c425036c0b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.063551 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4e626-83ea-469e-a54d-c425036c0b9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.064278 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4e626-83ea-469e-a54d-c425036c0b9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:59.064588 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4e626-83ea-469e-a54d-c425036c0b9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:59.066229 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4e626-83ea-469e-a54d-c425036c0b9b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:59.078282 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 764/1550] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:59.079524 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438bc1c-ded2-4916-ae96-8a867f58339b None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.079807 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438bc1c-ded2-4916-ae96-8a867f58339b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.080087 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438bc1c-ded2-4916-ae96-8a867f58339b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.086727 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 787/1551] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:43:58 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:59.091587 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79c0e9-56d2-44c4-bdb8-22ab01a5b548 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.091587 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79c0e9-56d2-44c4-bdb8-22ab01a5b548 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.091587 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79c0e9-56d2-44c4-bdb8-22ab01a5b548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.103152 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d438bc1c-ded2-4916-ae96-8a867f58339b None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:59.104373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d438bc1c-ded2-4916-ae96-8a867f58339b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:59.108991 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-8f79c0e9-56d2-44c4-bdb8-22ab01a5b548 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:59.110151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-8f79c0e9-56d2-44c4-bdb8-22ab01a5b548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:59.164619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 765/1552] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:59.166113 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed0eac-75cc-419d-b3ff-a11798d381a8 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.166404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed0eac-75cc-419d-b3ff-a11798d381a8 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.166719 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed0eac-75cc-419d-b3ff-a11798d381a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.179598 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 788/1553] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:59.180552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79c0e9-56d2-44c4-bdb8-22ab01a5b548 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:59.185769 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-4bed0eac-75cc-419d-b3ff-a11798d381a8 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:59.187542 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4bed0eac-75cc-419d-b3ff-a11798d381a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:59.190222 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79c0e9-56d2-44c4-bdb8-22ab01a5b548 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:59.194669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ece4d-5516-497c-a167-a672dccce6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:59.195328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ece4d-5516-497c-a167-a672dccce6d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.195613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ece4d-5516-497c-a167-a672dccce6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.195894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ece4d-5516-497c-a167-a672dccce6d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.196033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 766/1554] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:59.196586 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ece4d-5516-497c-a167-a672dccce6d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:59.197280 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4233c78-cc85-4aed-9e5f-00db21f06880 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.197567 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4233c78-cc85-4aed-9e5f-00db21f06880 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.197687 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: WARNING py.warnings [None req-bc2ece4d-5516-497c-a167-a672dccce6d2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:43:59.197687 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: warnings.warn(msg) Jun 23 11:43:59.197687 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: Jun 23 11:43:59.197945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4233c78-cc85-4aed-9e5f-00db21f06880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.198888 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ece4d-5516-497c-a167-a672dccce6d2 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:59.216701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a4233c78-cc85-4aed-9e5f-00db21f06880 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:59.218615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a4233c78-cc85-4aed-9e5f-00db21f06880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:59.227212 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 767/1555] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:59.228974 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e8b7d-4ef1-41f5-a320-fa848a439af8 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.229364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e8b7d-4ef1-41f5-a320-fa848a439af8 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.229864 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e8b7d-4ef1-41f5-a320-fa848a439af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.249430 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-6c5e8b7d-4ef1-41f5-a320-fa848a439af8 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:59.250651 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6c5e8b7d-4ef1-41f5-a320-fa848a439af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:59.258346 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 768/1556] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:59.259242 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e8b7d-4ef1-41f5-a320-fa848a439af8 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:59.269036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e8b7d-4ef1-41f5-a320-fa848a439af8 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:59.273536 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293e1b0-c4c0-4bbd-905f-aad2ee0b81e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:59.274169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293e1b0-c4c0-4bbd-905f-aad2ee0b81e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.274451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293e1b0-c4c0-4bbd-905f-aad2ee0b81e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.274766 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293e1b0-c4c0-4bbd-905f-aad2ee0b81e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.275435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293e1b0-c4c0-4bbd-905f-aad2ee0b81e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:59.277328 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293e1b0-c4c0-4bbd-905f-aad2ee0b81e8 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:59.289961 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 789/1557] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:43:59 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:59.290335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ece4d-5516-497c-a167-a672dccce6d2 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:59.303033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ece4d-5516-497c-a167-a672dccce6d2 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:59.308015 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1edf39-ac3a-4cfa-a84e-03303d407be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:59.308673 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1edf39-ac3a-4cfa-a84e-03303d407be1 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.308951 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1edf39-ac3a-4cfa-a84e-03303d407be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.309222 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1edf39-ac3a-4cfa-a84e-03303d407be1 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.309895 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1edf39-ac3a-4cfa-a84e-03303d407be1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:59.311863 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1edf39-ac3a-4cfa-a84e-03303d407be1 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:59.381876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 769/1558] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:43:59 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:59.383376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132dbc0c-f049-4a93-9a99-571030ac0dc7 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.383689 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132dbc0c-f049-4a93-9a99-571030ac0dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.383967 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132dbc0c-f049-4a93-9a99-571030ac0dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.401627 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-132dbc0c-f049-4a93-9a99-571030ac0dc7 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:59.402833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-132dbc0c-f049-4a93-9a99-571030ac0dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:59.408885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:59.412424 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 790/1559] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:43:59 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:59.412424 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff40fd0-adef-4413-9176-895b1f18ec84 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.412424 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff40fd0-adef-4413-9176-895b1f18ec84 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.412707 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff40fd0-adef-4413-9176-895b1f18ec84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.430588 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-2ff40fd0-adef-4413-9176-895b1f18ec84 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:59.431781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-2ff40fd0-adef-4413-9176-895b1f18ec84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:59.465802 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 770/1560] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:59.466806 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132dbc0c-f049-4a93-9a99-571030ac0dc7 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:59.478200 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:59.486580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132dbc0c-f049-4a93-9a99-571030ac0dc7 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:59.486580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01a6a4-2208-44c1-bc31-888473cbc6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:59.486580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01a6a4-2208-44c1-bc31-888473cbc6cb admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.486580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01a6a4-2208-44c1-bc31-888473cbc6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.487040 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01a6a4-2208-44c1-bc31-888473cbc6cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.487632 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01a6a4-2208-44c1-bc31-888473cbc6cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:59.488463 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01a6a4-2208-44c1-bc31-888473cbc6cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:59.490212 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01a6a4-2208-44c1-bc31-888473cbc6cb admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:59.493783 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 791/1561] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:59.495169 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f558b3e-5c45-44b5-91f1-1ccfc233d6ae None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.495521 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f558b3e-5c45-44b5-91f1-1ccfc233d6ae None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.495860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f558b3e-5c45-44b5-91f1-1ccfc233d6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.505133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 771/1562] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:59.506510 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f7b92-07b5-4c62-901a-e02ac3ad3429 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.506895 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f7b92-07b5-4c62-901a-e02ac3ad3429 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.507166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f7b92-07b5-4c62-901a-e02ac3ad3429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.516338 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-6f558b3e-5c45-44b5-91f1-1ccfc233d6ae None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:59.517537 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-6f558b3e-5c45-44b5-91f1-1ccfc233d6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:59.525398 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 792/1563] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:59.526188 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f558b3e-5c45-44b5-91f1-1ccfc233d6ae None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:59.535289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-cb0f7b92-07b5-4c62-901a-e02ac3ad3429 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:59.538312 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f558b3e-5c45-44b5-91f1-1ccfc233d6ae None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:59.538480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-cb0f7b92-07b5-4c62-901a-e02ac3ad3429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:59.539972 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb7e1f-5d13-4b5d-8f21-60cef53a8b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:59.540615 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb7e1f-5d13-4b5d-8f21-60cef53a8b0b admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.540894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb7e1f-5d13-4b5d-8f21-60cef53a8b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.541169 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb7e1f-5d13-4b5d-8f21-60cef53a8b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.541873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb7e1f-5d13-4b5d-8f21-60cef53a8b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:59.543484 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb7e1f-5d13-4b5d-8f21-60cef53a8b0b admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:59.544362 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 772/1564] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:59.545184 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f7b92-07b5-4c62-901a-e02ac3ad3429 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:59.554395 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f7b92-07b5-4c62-901a-e02ac3ad3429 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:59.559189 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423d201-9cc8-4965-af3d-22182d75fe67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:59.559891 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423d201-9cc8-4965-af3d-22182d75fe67 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.560164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423d201-9cc8-4965-af3d-22182d75fe67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.560437 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423d201-9cc8-4965-af3d-22182d75fe67 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.561178 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 793/1565] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:43:59.561800 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423d201-9cc8-4965-af3d-22182d75fe67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:59.562280 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb7e1f-5d13-4b5d-8f21-60cef53a8b0b admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:59.564041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423d201-9cc8-4965-af3d-22182d75fe67 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:59.575683 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb7e1f-5d13-4b5d-8f21-60cef53a8b0b admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:59.581241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d32f9e-39d4-4d30-aa5e-4d8824529573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:59.581892 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d32f9e-39d4-4d30-aa5e-4d8824529573 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.582172 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d32f9e-39d4-4d30-aa5e-4d8824529573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.582437 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d32f9e-39d4-4d30-aa5e-4d8824529573 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.583186 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d32f9e-39d4-4d30-aa5e-4d8824529573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:59.585017 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d32f9e-39d4-4d30-aa5e-4d8824529573 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:59.595362 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 794/1566] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:43:59.596161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d32f9e-39d4-4d30-aa5e-4d8824529573 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:59.605496 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d32f9e-39d4-4d30-aa5e-4d8824529573 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:59.610171 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f2464-38d4-46a2-95dd-cedb030708e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:59.610824 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f2464-38d4-46a2-95dd-cedb030708e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.611113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f2464-38d4-46a2-95dd-cedb030708e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.611395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f2464-38d4-46a2-95dd-cedb030708e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.612102 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f2464-38d4-46a2-95dd-cedb030708e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:59.612409 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f2464-38d4-46a2-95dd-cedb030708e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:59.613962 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f2464-38d4-46a2-95dd-cedb030708e7 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:59.632428 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 795/1567] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:59.632428 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f2464-38d4-46a2-95dd-cedb030708e7 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:59.636914 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f2464-38d4-46a2-95dd-cedb030708e7 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:59.642168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff296ce6-b829-423e-9079-a4a93da80e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:59.642827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff296ce6-b829-423e-9079-a4a93da80e98 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.643110 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff296ce6-b829-423e-9079-a4a93da80e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.643386 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff296ce6-b829-423e-9079-a4a93da80e98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.644134 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff296ce6-b829-423e-9079-a4a93da80e98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:59.644442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff296ce6-b829-423e-9079-a4a93da80e98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:59.645990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff296ce6-b829-423e-9079-a4a93da80e98 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:59.657178 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 796/1568] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:59.658925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff296ce6-b829-423e-9079-a4a93da80e98 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:59.667548 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff296ce6-b829-423e-9079-a4a93da80e98 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:59.672616 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ed0d6-50a5-4708-a19e-ab47cf5497ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:59.673307 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ed0d6-50a5-4708-a19e-ab47cf5497ea admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.673677 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ed0d6-50a5-4708-a19e-ab47cf5497ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.673999 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ed0d6-50a5-4708-a19e-ab47cf5497ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.674705 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ed0d6-50a5-4708-a19e-ab47cf5497ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:59.676583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ed0d6-50a5-4708-a19e-ab47cf5497ea admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:59.684712 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 773/1569] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:43:59 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:59.684712 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423d201-9cc8-4965-af3d-22182d75fe67 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:59.694886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423d201-9cc8-4965-af3d-22182d75fe67 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:59.699595 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75887903-7bdc-4eb5-be24-c142e2ca0b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:59.700242 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75887903-7bdc-4eb5-be24-c142e2ca0b8f admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.700524 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75887903-7bdc-4eb5-be24-c142e2ca0b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.700797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75887903-7bdc-4eb5-be24-c142e2ca0b8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.701514 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75887903-7bdc-4eb5-be24-c142e2ca0b8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:59.701820 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75887903-7bdc-4eb5-be24-c142e2ca0b8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:59.704314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75887903-7bdc-4eb5-be24-c142e2ca0b8f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:59.716832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 774/1570] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:59.717995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75887903-7bdc-4eb5-be24-c142e2ca0b8f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:59.727723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75887903-7bdc-4eb5-be24-c142e2ca0b8f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:59.732185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ded3d-bee7-4604-977a-48197d03c51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:59.732797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ded3d-bee7-4604-977a-48197d03c51d admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.733082 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ded3d-bee7-4604-977a-48197d03c51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.733521 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ded3d-bee7-4604-977a-48197d03c51d admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.734253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ded3d-bee7-4604-977a-48197d03c51d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:59.734588 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ded3d-bee7-4604-977a-48197d03c51d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:59.736192 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ded3d-bee7-4604-977a-48197d03c51d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:59.747380 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 775/1571] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:59.748660 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ded3d-bee7-4604-977a-48197d03c51d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:59.758148 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ded3d-bee7-4604-977a-48197d03c51d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:59.762944 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f4573-b8f4-4793-9de4-9aecb8e532c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:59.763590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f4573-b8f4-4793-9de4-9aecb8e532c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.763871 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f4573-b8f4-4793-9de4-9aecb8e532c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.764146 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f4573-b8f4-4793-9de4-9aecb8e532c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.765613 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f4573-b8f4-4793-9de4-9aecb8e532c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:59.767197 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f4573-b8f4-4793-9de4-9aecb8e532c3 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:59.777515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 776/1572] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:43:59.778320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f4573-b8f4-4793-9de4-9aecb8e532c3 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:59.787138 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 797/1573] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:43:59 2022] POST /identity/v3/projects => generated 391 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:59.787138 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ed0d6-50a5-4708-a19e-ab47cf5497ea admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:59.791610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f4573-b8f4-4793-9de4-9aecb8e532c3 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:59.845234 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ed0d6-50a5-4708-a19e-ab47cf5497ea admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:59.849255 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e41910-5169-4e81-8801-11f8c8f766ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:59.849979 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e41910-5169-4e81-8801-11f8c8f766ec admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.850267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e41910-5169-4e81-8801-11f8c8f766ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.850366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1795160-e05a-41d5-9907-92c44fc4a206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:59.850557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e41910-5169-4e81-8801-11f8c8f766ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.851149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1795160-e05a-41d5-9907-92c44fc4a206 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.851295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e41910-5169-4e81-8801-11f8c8f766ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:59.851567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1795160-e05a-41d5-9907-92c44fc4a206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.851914 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1795160-e05a-41d5-9907-92c44fc4a206 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.852681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1795160-e05a-41d5-9907-92c44fc4a206 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:59.852932 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e41910-5169-4e81-8801-11f8c8f766ec admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:59.854791 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1795160-e05a-41d5-9907-92c44fc4a206 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:59.866474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 777/1574] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:43:59.866474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e41910-5169-4e81-8801-11f8c8f766ec admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:59.876856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e41910-5169-4e81-8801-11f8c8f766ec admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:59.881540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eeca7d-bd3d-4a95-bb7e-471625a6b6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:59.882177 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeca7d-bd3d-4a95-bb7e-471625a6b6ac admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:59.882512 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeca7d-bd3d-4a95-bb7e-471625a6b6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:59.882779 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeca7d-bd3d-4a95-bb7e-471625a6b6ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:59.883469 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eeca7d-bd3d-4a95-bb7e-471625a6b6ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:59.885374 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eeca7d-bd3d-4a95-bb7e-471625a6b6ac admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:59.972657 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 798/1575] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:43:59 2022] POST /identity/v3/projects => generated 379 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:59.973658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1795160-e05a-41d5-9907-92c44fc4a206 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:59.988416 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 778/1576] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:43:59 2022] POST /identity/v3/projects => generated 363 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:59.988416 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eeca7d-bd3d-4a95-bb7e-471625a6b6ac admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:59.998442 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:00.037177 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1795160-e05a-41d5-9907-92c44fc4a206 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:00.040907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:00.049654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eeca7d-bd3d-4a95-bb7e-471625a6b6ac admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:00.054833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89cc633-b1b7-4a3e-b2ba-2eaf1543c998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:00.055338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89cc633-b1b7-4a3e-b2ba-2eaf1543c998 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:00.056412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89cc633-b1b7-4a3e-b2ba-2eaf1543c998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:00.056412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89cc633-b1b7-4a3e-b2ba-2eaf1543c998 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:00.057096 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89cc633-b1b7-4a3e-b2ba-2eaf1543c998 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:00.059046 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89cc633-b1b7-4a3e-b2ba-2eaf1543c998 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:00.093677 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57809e48-e12c-47a0-a6a8-b9ff855ffab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:00.094352 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57809e48-e12c-47a0-a6a8-b9ff855ffab5 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:00.094666 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57809e48-e12c-47a0-a6a8-b9ff855ffab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:00.094941 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57809e48-e12c-47a0-a6a8-b9ff855ffab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:00.095656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57809e48-e12c-47a0-a6a8-b9ff855ffab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:00.097247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57809e48-e12c-47a0-a6a8-b9ff855ffab5 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:00.106713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 799/1577] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:00.107462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57809e48-e12c-47a0-a6a8-b9ff855ffab5 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:00.117256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57809e48-e12c-47a0-a6a8-b9ff855ffab5 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:00.123135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505634a-ecdf-4809-9083-da49ddd453de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:00.123135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505634a-ecdf-4809-9083-da49ddd453de admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:00.123135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505634a-ecdf-4809-9083-da49ddd453de admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:00.123135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505634a-ecdf-4809-9083-da49ddd453de admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:00.123569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505634a-ecdf-4809-9083-da49ddd453de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:00.126039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505634a-ecdf-4809-9083-da49ddd453de admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:00.153132 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 779/1578] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:43:59 2022] POST /identity/v3/projects => generated 405 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:00.153132 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89cc633-b1b7-4a3e-b2ba-2eaf1543c998 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:00.213544 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89cc633-b1b7-4a3e-b2ba-2eaf1543c998 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:00.218984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6e52e-ae94-41d0-b7d2-7bc1d263dc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:00.219870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6e52e-ae94-41d0-b7d2-7bc1d263dc12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:00.220158 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6e52e-ae94-41d0-b7d2-7bc1d263dc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:00.220441 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6e52e-ae94-41d0-b7d2-7bc1d263dc12 admin admin] PATH_INFO: `/v3/projects/0252f30c9ada49a081547315cb6dcc22/users/40552f9345a34e86b8a0a791bef6b7bb/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:00.235613 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd6e52e-ae94-41d0-b7d2-7bc1d263dc12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0252f30c9ada49a081547315cb6dcc22, user_id=40552f9345a34e86b8a0a791bef6b7bb, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:00.236682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: WARNING py.warnings [None req-dbd6e52e-ae94-41d0-b7d2-7bc1d263dc12 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:44:00.236682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: warnings.warn(msg) Jun 23 11:44:00.236682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: Jun 23 11:44:00.238751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd6e52e-ae94-41d0-b7d2-7bc1d263dc12 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:00.242526 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 800/1579] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:00 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:00.246310 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505634a-ecdf-4809-9083-da49ddd453de admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:00.307004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505634a-ecdf-4809-9083-da49ddd453de admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:00.308632 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 780/1580] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:00 2022] PUT /identity/v3/projects/0252f30c9ada49a081547315cb6dcc22/users/40552f9345a34e86b8a0a791bef6b7bb/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:00.309431 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6e52e-ae94-41d0-b7d2-7bc1d263dc12 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:00.371290 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6e52e-ae94-41d0-b7d2-7bc1d263dc12 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:00.372894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5fff4-a508-4415-813b-764541062f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:00.373566 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5fff4-a508-4415-813b-764541062f7c admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:00.373852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5fff4-a508-4415-813b-764541062f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:00.374128 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5fff4-a508-4415-813b-764541062f7c admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:00.374859 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a5fff4-a508-4415-813b-764541062f7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:00.376087 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2dfe3-7043-49a3-b312-b8539a4931a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:00.376796 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a5fff4-a508-4415-813b-764541062f7c admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:00.376974 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2dfe3-7043-49a3-b312-b8539a4931a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:00.377261 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2dfe3-7043-49a3-b312-b8539a4931a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:00.377538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2dfe3-7043-49a3-b312-b8539a4931a4 admin admin] PATH_INFO: `/v3/projects/ee135767dfc846469d64b658372994a8/users/b7fd8323d29d445f903e2103453e8ae6/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:00.391959 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2dfe3-7043-49a3-b312-b8539a4931a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee135767dfc846469d64b658372994a8, user_id=b7fd8323d29d445f903e2103453e8ae6, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:00.394828 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2dfe3-7043-49a3-b312-b8539a4931a4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:00.486626 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 801/1581] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:00 2022] POST /identity/v3/users => generated 296 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:00.487749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5fff4-a508-4415-813b-764541062f7c admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:00.498344 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5fff4-a508-4415-813b-764541062f7c admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:00.503422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b280053-a065-4439-b5fd-844729a6b2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:00.504110 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b280053-a065-4439-b5fd-844729a6b2a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:00.512512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b280053-a065-4439-b5fd-844729a6b2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:00.512512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b280053-a065-4439-b5fd-844729a6b2a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:00.512512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b280053-a065-4439-b5fd-844729a6b2a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:00.512512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b280053-a065-4439-b5fd-844729a6b2a5 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:00.512837 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 781/1582] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:00 2022] PUT /identity/v3/projects/ee135767dfc846469d64b658372994a8/users/b7fd8323d29d445f903e2103453e8ae6/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:00.512837 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2dfe3-7043-49a3-b312-b8539a4931a4 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:00.514032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:00.572570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2dfe3-7043-49a3-b312-b8539a4931a4 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:00.578022 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c232fcf-7180-4927-86b3-24d259fbbdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:00.578729 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c232fcf-7180-4927-86b3-24d259fbbdda admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:00.579018 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c232fcf-7180-4927-86b3-24d259fbbdda admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:00.579298 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c232fcf-7180-4927-86b3-24d259fbbdda admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:00.580017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c232fcf-7180-4927-86b3-24d259fbbdda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:00.582012 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c232fcf-7180-4927-86b3-24d259fbbdda admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:00.623762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:00.629260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 802/1583] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:00 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:00.630052 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b280053-a065-4439-b5fd-844729a6b2a5 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:00.639345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b280053-a065-4439-b5fd-844729a6b2a5 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:00.644487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606614d-37a2-4f5e-bb94-fe70434f4362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:00.645118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606614d-37a2-4f5e-bb94-fe70434f4362 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:00.645403 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606614d-37a2-4f5e-bb94-fe70434f4362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:00.645682 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606614d-37a2-4f5e-bb94-fe70434f4362 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:00.646394 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606614d-37a2-4f5e-bb94-fe70434f4362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:00.648025 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606614d-37a2-4f5e-bb94-fe70434f4362 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:00.659222 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 803/1584] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:00.661902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606614d-37a2-4f5e-bb94-fe70434f4362 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:00.669367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606614d-37a2-4f5e-bb94-fe70434f4362 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:00.674141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123b7aa-4250-4ca2-be11-2059e87f11c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:00.674802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123b7aa-4250-4ca2-be11-2059e87f11c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:00.675080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123b7aa-4250-4ca2-be11-2059e87f11c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:00.675360 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123b7aa-4250-4ca2-be11-2059e87f11c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:00.676028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123b7aa-4250-4ca2-be11-2059e87f11c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:00.677893 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123b7aa-4250-4ca2-be11-2059e87f11c9 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:00.688969 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 782/1585] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:00 2022] POST /identity/v3/users => generated 282 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:00.688969 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c232fcf-7180-4927-86b3-24d259fbbdda admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:00.696766 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c232fcf-7180-4927-86b3-24d259fbbdda admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:00.701344 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3597b-4404-4d50-a68c-51be617ab96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:00.702389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3597b-4404-4d50-a68c-51be617ab96e admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:00.702973 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3597b-4404-4d50-a68c-51be617ab96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:00.703450 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3597b-4404-4d50-a68c-51be617ab96e admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:00.704193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3597b-4404-4d50-a68c-51be617ab96e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:00.706518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3597b-4404-4d50-a68c-51be617ab96e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:00.766296 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 804/1586] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:00 2022] POST /identity/v3/users => generated 303 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:00.767627 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c1a88-389e-4a30-a94a-c2902b195b56 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:00.767921 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c1a88-389e-4a30-a94a-c2902b195b56 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:00.768234 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c1a88-389e-4a30-a94a-c2902b195b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:00.795669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-4d6c1a88-389e-4a30-a94a-c2902b195b56 None None] MFA Rules not processed for user `40552f9345a34e86b8a0a791bef6b7bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:00.797763 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 783/1587] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:00 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:00.797763 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3597b-4404-4d50-a68c-51be617ab96e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:00.797961 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-4d6c1a88-389e-4a30-a94a-c2902b195b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:00.808490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3597b-4404-4d50-a68c-51be617ab96e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:00.813291 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413df52-20cd-4978-a0a6-947cf70cb47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:00.813944 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413df52-20cd-4978-a0a6-947cf70cb47d admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:00.814255 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413df52-20cd-4978-a0a6-947cf70cb47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:00.814556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413df52-20cd-4978-a0a6-947cf70cb47d admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:00.815285 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413df52-20cd-4978-a0a6-947cf70cb47d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:00.816973 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413df52-20cd-4978-a0a6-947cf70cb47d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:00.828201 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 784/1588] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:00.829730 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816145cd-c5cb-4ca1-98ee-2ef8c354137a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:00.830016 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816145cd-c5cb-4ca1-98ee-2ef8c354137a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:00.830300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816145cd-c5cb-4ca1-98ee-2ef8c354137a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:00.848094 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-816145cd-c5cb-4ca1-98ee-2ef8c354137a None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:00.849493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-816145cd-c5cb-4ca1-98ee-2ef8c354137a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:00.857150 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 785/1589] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:00.857963 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816145cd-c5cb-4ca1-98ee-2ef8c354137a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:00.866579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 805/1590] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:00.867405 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816145cd-c5cb-4ca1-98ee-2ef8c354137a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:00.867891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c1a88-389e-4a30-a94a-c2902b195b56 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:00.875048 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f639296-7c0d-4ffd-a168-c75e02fa00ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:00.875722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f639296-7c0d-4ffd-a168-c75e02fa00ba admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:00.876532 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f639296-7c0d-4ffd-a168-c75e02fa00ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:00.876532 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f639296-7c0d-4ffd-a168-c75e02fa00ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:00.876989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f639296-7c0d-4ffd-a168-c75e02fa00ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:00.878599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f639296-7c0d-4ffd-a168-c75e02fa00ba admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:00.880731 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c1a88-389e-4a30-a94a-c2902b195b56 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:00.886223 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261a6c3-693e-4c49-9345-5fe85efa3eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:00.887139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261a6c3-693e-4c49-9345-5fe85efa3eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:00.887422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261a6c3-693e-4c49-9345-5fe85efa3eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:00.887694 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261a6c3-693e-4c49-9345-5fe85efa3eb5 admin admin] PATH_INFO: `/v3/projects/4431791e3ce94f4cae4d648f1af2f186/users/72cb75b4931b42f6bd5f71fa0a99a15e/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:00.894074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 786/1591] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:00.895180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f639296-7c0d-4ffd-a168-c75e02fa00ba admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:00.907930 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261a6c3-693e-4c49-9345-5fe85efa3eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4431791e3ce94f4cae4d648f1af2f186, user_id=72cb75b4931b42f6bd5f71fa0a99a15e, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:00.912318 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f639296-7c0d-4ffd-a168-c75e02fa00ba admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:00.912452 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261a6c3-693e-4c49-9345-5fe85efa3eb5 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:00.914195 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78070413-5c38-4767-ba87-5da706641fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:00.915043 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78070413-5c38-4767-ba87-5da706641fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:00.915352 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78070413-5c38-4767-ba87-5da706641fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:00.915661 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78070413-5c38-4767-ba87-5da706641fb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:00.916445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78070413-5c38-4767-ba87-5da706641fb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:00.918118 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78070413-5c38-4767-ba87-5da706641fb6 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:00.928507 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 787/1592] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:00.929291 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78070413-5c38-4767-ba87-5da706641fb6 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:00.939109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78070413-5c38-4767-ba87-5da706641fb6 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:00.943715 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833a5f1-ec78-4ebe-a198-db3402fa5445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:00.944335 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833a5f1-ec78-4ebe-a198-db3402fa5445 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:00.944619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833a5f1-ec78-4ebe-a198-db3402fa5445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:00.944894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833a5f1-ec78-4ebe-a198-db3402fa5445 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:00.945594 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833a5f1-ec78-4ebe-a198-db3402fa5445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:00.947256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833a5f1-ec78-4ebe-a198-db3402fa5445 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:00.958201 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 788/1593] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:00.959304 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833a5f1-ec78-4ebe-a198-db3402fa5445 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:00.968510 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833a5f1-ec78-4ebe-a198-db3402fa5445 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:00.973156 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f458f-97f8-492c-b78a-70c4040c6aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:00.973759 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f458f-97f8-492c-b78a-70c4040c6aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:00.974043 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f458f-97f8-492c-b78a-70c4040c6aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:00.974315 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f458f-97f8-492c-b78a-70c4040c6aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:00.975038 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f458f-97f8-492c-b78a-70c4040c6aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:00.976554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f458f-97f8-492c-b78a-70c4040c6aa2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:00.986714 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 789/1594] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:00.987534 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f458f-97f8-492c-b78a-70c4040c6aa2 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:00.996691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f458f-97f8-492c-b78a-70c4040c6aa2 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:00.998089 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 806/1595] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:00 2022] PUT /identity/v3/projects/4431791e3ce94f4cae4d648f1af2f186/users/72cb75b4931b42f6bd5f71fa0a99a15e/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:01.001469 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261a6c3-693e-4c49-9345-5fe85efa3eb5 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:01.056201 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261a6c3-693e-4c49-9345-5fe85efa3eb5 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:01.056783 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3ad98-5c9f-4a3f-8af4-a7f4a0082d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:01.057458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3ad98-5c9f-4a3f-8af4-a7f4a0082d62 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.057740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3ad98-5c9f-4a3f-8af4-a7f4a0082d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.058014 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3ad98-5c9f-4a3f-8af4-a7f4a0082d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:01.058825 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3ad98-5c9f-4a3f-8af4-a7f4a0082d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:01.060074 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74240da-93cd-41bc-832c-21f6b524fad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:01.060445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3ad98-5c9f-4a3f-8af4-a7f4a0082d62 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:01.060729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74240da-93cd-41bc-832c-21f6b524fad7 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.061015 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74240da-93cd-41bc-832c-21f6b524fad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.061893 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74240da-93cd-41bc-832c-21f6b524fad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:01.071356 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74240da-93cd-41bc-832c-21f6b524fad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:01.076082 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 790/1596] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:01.076305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74240da-93cd-41bc-832c-21f6b524fad7 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:01.076406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a34b89-6827-437e-9b4d-b6dc660652bb None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.076406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a34b89-6827-437e-9b4d-b6dc660652bb None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.076406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a34b89-6827-437e-9b4d-b6dc660652bb None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:01.076406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 791/1597] 10.222.0.13 () {54 vars in 961 bytes} [Thu Jun 23 11:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:44:01.076406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a34b89-6827-437e-9b4d-b6dc660652bb None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:01.084479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 807/1598] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:01.085777 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74240da-93cd-41bc-832c-21f6b524fad7 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:01.091070 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a34b89-6827-437e-9b4d-b6dc660652bb None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:01.096001 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74240da-93cd-41bc-832c-21f6b524fad7 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:01.096887 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c48e4f-c9a2-4950-8e8d-2c2941a9b3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:01.097605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c48e4f-c9a2-4950-8e8d-2c2941a9b3dc admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.097990 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c48e4f-c9a2-4950-8e8d-2c2941a9b3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.098266 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c48e4f-c9a2-4950-8e8d-2c2941a9b3dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:01.099396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c48e4f-c9a2-4950-8e8d-2c2941a9b3dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:01.101314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c48e4f-c9a2-4950-8e8d-2c2941a9b3dc admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:01.101450 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024c9fd-e0a0-4d72-8658-4ae0a4bee677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:01.101669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024c9fd-e0a0-4d72-8658-4ae0a4bee677 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.101953 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024c9fd-e0a0-4d72-8658-4ae0a4bee677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.102361 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024c9fd-e0a0-4d72-8658-4ae0a4bee677 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:01.102991 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0024c9fd-e0a0-4d72-8658-4ae0a4bee677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:01.103984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:01.104603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0024c9fd-e0a0-4d72-8658-4ae0a4bee677 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:01.115052 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 792/1599] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:01.116259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c48e4f-c9a2-4950-8e8d-2c2941a9b3dc admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:01.117886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 808/1600] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:01.118925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024c9fd-e0a0-4d72-8658-4ae0a4bee677 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:01.127186 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c48e4f-c9a2-4950-8e8d-2c2941a9b3dc admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:01.129885 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024c9fd-e0a0-4d72-8658-4ae0a4bee677 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:01.131935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c05475-3136-4b0c-ad37-b6b83bbe4207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:01.132587 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c05475-3136-4b0c-ad37-b6b83bbe4207 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.132868 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c05475-3136-4b0c-ad37-b6b83bbe4207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.133314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c05475-3136-4b0c-ad37-b6b83bbe4207 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:01.134072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c05475-3136-4b0c-ad37-b6b83bbe4207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:01.134609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee5697-6f63-42fe-a5b9-3d8f0477dd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:01.135269 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee5697-6f63-42fe-a5b9-3d8f0477dd5e admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.135564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee5697-6f63-42fe-a5b9-3d8f0477dd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.135724 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c05475-3136-4b0c-ad37-b6b83bbe4207 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:01.135946 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee5697-6f63-42fe-a5b9-3d8f0477dd5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:01.136670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee5697-6f63-42fe-a5b9-3d8f0477dd5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:01.138271 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee5697-6f63-42fe-a5b9-3d8f0477dd5e admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:01.146448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 793/1601] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:01.147705 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc60a76-236b-469b-973d-0bfb303fe073 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.147992 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc60a76-236b-469b-973d-0bfb303fe073 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.148265 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc60a76-236b-469b-973d-0bfb303fe073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:01.156594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 809/1602] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:01.156594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee5697-6f63-42fe-a5b9-3d8f0477dd5e admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:01.159664 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee5697-6f63-42fe-a5b9-3d8f0477dd5e admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:01.164664 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd541beb-946c-4474-9ec1-08958dc7d543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:01.165526 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd541beb-946c-4474-9ec1-08958dc7d543 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.165807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd541beb-946c-4474-9ec1-08958dc7d543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.166116 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd541beb-946c-4474-9ec1-08958dc7d543 admin admin] PATH_INFO: `/v3/projects/b6241924efa944a68503a87b695c7410/users/bcd5fc6167654972868941ed28c327b6/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:01.175587 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-bfc60a76-236b-469b-973d-0bfb303fe073 None None] MFA Rules not processed for user `72cb75b4931b42f6bd5f71fa0a99a15e`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:01.176917 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bfc60a76-236b-469b-973d-0bfb303fe073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:01.181994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd541beb-946c-4474-9ec1-08958dc7d543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6241924efa944a68503a87b695c7410, user_id=bcd5fc6167654972868941ed28c327b6, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:01.184759 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd541beb-946c-4474-9ec1-08958dc7d543 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:01.187610 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:01.249325 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 794/1603] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:01.251244 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec064f38-80dc-402b-ac26-fad2f70f4d4f None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.251725 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec064f38-80dc-402b-ac26-fad2f70f4d4f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.251725 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec064f38-80dc-402b-ac26-fad2f70f4d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:01.270579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 810/1604] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:01 2022] PUT /identity/v3/projects/b6241924efa944a68503a87b695c7410/users/bcd5fc6167654972868941ed28c327b6/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:01.270579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd541beb-946c-4474-9ec1-08958dc7d543 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:01.272381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ec064f38-80dc-402b-ac26-fad2f70f4d4f None None] MFA Rules not processed for user `161c5ed1e47e450290a1f428ee100b68`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:01.273734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ec064f38-80dc-402b-ac26-fad2f70f4d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:01.346980 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 795/1605] 10.222.0.13 () {58 vars in 1058 bytes} [Thu Jun 23 11:44:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:01.347892 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec064f38-80dc-402b-ac26-fad2f70f4d4f None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:01.352278 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd541beb-946c-4474-9ec1-08958dc7d543 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:01.357359 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab662d-5938-4359-9fe4-46cd1747798e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:01.357977 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec064f38-80dc-402b-ac26-fad2f70f4d4f None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:01.359081 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab662d-5938-4359-9fe4-46cd1747798e admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.359421 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab662d-5938-4359-9fe4-46cd1747798e admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.359639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab662d-5938-4359-9fe4-46cd1747798e admin admin] PATH_INFO: `/v3/projects/515e65a705ee46cc8f8e03efbe58f177/users/ae72d6c494ba4f16989adfe090803d47/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:01.363420 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91a7d8-372f-4e13-ac8d-8bf52e1bde12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:01.364289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91a7d8-372f-4e13-ac8d-8bf52e1bde12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.364571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91a7d8-372f-4e13-ac8d-8bf52e1bde12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.364843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91a7d8-372f-4e13-ac8d-8bf52e1bde12 admin admin] PATH_INFO: `/v3/projects/96a5cce3a55c4f819529ccb1cadb2b6a/users/b4f455bc358f4faaa702c96d4818aeb2/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:01.375758 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab662d-5938-4359-9fe4-46cd1747798e admin admin] RBAC: Authorizing `identity:create_grant(project_id=515e65a705ee46cc8f8e03efbe58f177, user_id=ae72d6c494ba4f16989adfe090803d47, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:01.378777 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab662d-5938-4359-9fe4-46cd1747798e admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:01.383212 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91a7d8-372f-4e13-ac8d-8bf52e1bde12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96a5cce3a55c4f819529ccb1cadb2b6a, user_id=b4f455bc358f4faaa702c96d4818aeb2, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:01.393778 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91a7d8-372f-4e13-ac8d-8bf52e1bde12 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:01.461525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 796/1606] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:01 2022] PUT /identity/v3/projects/96a5cce3a55c4f819529ccb1cadb2b6a/users/b4f455bc358f4faaa702c96d4818aeb2/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:01.462119 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91a7d8-372f-4e13-ac8d-8bf52e1bde12 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:01.473691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 811/1607] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:01 2022] PUT /identity/v3/projects/515e65a705ee46cc8f8e03efbe58f177/users/ae72d6c494ba4f16989adfe090803d47/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:01.473691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab662d-5938-4359-9fe4-46cd1747798e admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:01.534195 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:01.552865 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91a7d8-372f-4e13-ac8d-8bf52e1bde12 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:01.555647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab662d-5938-4359-9fe4-46cd1747798e admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:01.557750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25ad39-93f0-47fe-900e-f2561742eb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:01.558743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25ad39-93f0-47fe-900e-f2561742eb02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.559089 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25ad39-93f0-47fe-900e-f2561742eb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.559369 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25ad39-93f0-47fe-900e-f2561742eb02 admin admin] PATH_INFO: `/v3/projects/e33164546fd046309332f7e8b447dfb0/users/f13429cf5e49438b987a49564c345e3d/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:01.560414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc42cc-d964-4fed-8af2-8d9397e188fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:01.561957 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc42cc-d964-4fed-8af2-8d9397e188fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.562381 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc42cc-d964-4fed-8af2-8d9397e188fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.562793 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc42cc-d964-4fed-8af2-8d9397e188fe admin admin] PATH_INFO: `/v3/projects/45db495ff8c04b55b44c80a3ab412fb9/users/d1b988f3241a46308b7c3683e2943b78/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:01.574469 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e25ad39-93f0-47fe-900e-f2561742eb02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e33164546fd046309332f7e8b447dfb0, user_id=f13429cf5e49438b987a49564c345e3d, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:01.577197 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc42cc-d964-4fed-8af2-8d9397e188fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=45db495ff8c04b55b44c80a3ab412fb9, user_id=d1b988f3241a46308b7c3683e2943b78, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:01.577679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e25ad39-93f0-47fe-900e-f2561742eb02 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:01.580056 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc42cc-d964-4fed-8af2-8d9397e188fe admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:01.667305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 797/1608] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:01 2022] PUT /identity/v3/projects/e33164546fd046309332f7e8b447dfb0/users/f13429cf5e49438b987a49564c345e3d/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:01.668687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb918d08-78c6-4ee9-bad9-1151c58b5910 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.669007 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb918d08-78c6-4ee9-bad9-1151c58b5910 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.669287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb918d08-78c6-4ee9-bad9-1151c58b5910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:01.687560 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-cb918d08-78c6-4ee9-bad9-1151c58b5910 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:01.688765 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-cb918d08-78c6-4ee9-bad9-1151c58b5910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:01.704284 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 812/1609] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:01 2022] PUT /identity/v3/projects/45db495ff8c04b55b44c80a3ab412fb9/users/d1b988f3241a46308b7c3683e2943b78/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:01.705478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ff967-4c51-4595-a4a8-7c192b027a64 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.705771 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ff967-4c51-4595-a4a8-7c192b027a64 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.706049 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ff967-4c51-4595-a4a8-7c192b027a64 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:01.707580 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 813/1610] 10.222.0.13 () {54 vars in 961 bytes} [Thu Jun 23 11:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:44:01.708330 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ff967-4c51-4595-a4a8-7c192b027a64 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:01.777071 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 798/1611] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:01.778837 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92352a30-1ece-41de-be31-9995ee529d9e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.779132 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92352a30-1ece-41de-be31-9995ee529d9e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.779409 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92352a30-1ece-41de-be31-9995ee529d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:01.789112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:01.806284 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ff967-4c51-4595-a4a8-7c192b027a64 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:01.807956 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-92352a30-1ece-41de-be31-9995ee529d9e None None] MFA Rules not processed for user `b4f455bc358f4faaa702c96d4818aeb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:01.809174 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-92352a30-1ece-41de-be31-9995ee529d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:01.812328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534cd92-18bc-4073-a417-8e575f0da4ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:01.813010 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534cd92-18bc-4073-a417-8e575f0da4ca service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.813300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534cd92-18bc-4073-a417-8e575f0da4ca service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.813579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534cd92-18bc-4073-a417-8e575f0da4ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:01.880613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534cd92-18bc-4073-a417-8e575f0da4ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:01.882125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 799/1612] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:01 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:01.882451 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534cd92-18bc-4073-a417-8e575f0da4ca service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:01.883831 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25edfa8-113a-4216-985d-b42e612fa2da None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.884331 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25edfa8-113a-4216-985d-b42e612fa2da None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.884746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25edfa8-113a-4216-985d-b42e612fa2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:01.894111 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 814/1613] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:01.895455 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454c9c9-564b-4bae-adf8-3fb1316ca64b None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.895747 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454c9c9-564b-4bae-adf8-3fb1316ca64b None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.896029 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454c9c9-564b-4bae-adf8-3fb1316ca64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:01.912713 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f25edfa8-113a-4216-985d-b42e612fa2da None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:01.914269 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f25edfa8-113a-4216-985d-b42e612fa2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:01.915330 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-5454c9c9-564b-4bae-adf8-3fb1316ca64b None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:01.916560 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-5454c9c9-564b-4bae-adf8-3fb1316ca64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:01.984987 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:01.991855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 815/1614] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:01.993336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916bad5-9b24-4d74-91ea-98c0ae7b1dc5 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:01.993694 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916bad5-9b24-4d74-91ea-98c0ae7b1dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:01.994205 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916bad5-9b24-4d74-91ea-98c0ae7b1dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:02.002276 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 800/1615] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:02.003737 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559e0c8-9b1a-412f-b199-f2c91a3c0ab8 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:02.004096 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559e0c8-9b1a-412f-b199-f2c91a3c0ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:02.004508 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559e0c8-9b1a-412f-b199-f2c91a3c0ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:02.019742 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-b916bad5-9b24-4d74-91ea-98c0ae7b1dc5 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:02.021352 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b916bad5-9b24-4d74-91ea-98c0ae7b1dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:02.024120 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-4559e0c8-9b1a-412f-b199-f2c91a3c0ab8 None None] MFA Rules not processed for user `161c5ed1e47e450290a1f428ee100b68`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:02.025323 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4559e0c8-9b1a-412f-b199-f2c91a3c0ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:02.031268 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 816/1616] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:02.032337 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916bad5-9b24-4d74-91ea-98c0ae7b1dc5 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:02.045549 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 801/1617] 10.222.0.13 () {58 vars in 1058 bytes} [Thu Jun 23 11:44:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:02.053807 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559e0c8-9b1a-412f-b199-f2c91a3c0ab8 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:02.064575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559e0c8-9b1a-412f-b199-f2c91a3c0ab8 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:02.070054 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3c269-c2b0-4d1d-9d1d-087dcc2dc178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:02.071784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3c269-c2b0-4d1d-9d1d-087dcc2dc178 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:02.071784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3c269-c2b0-4d1d-9d1d-087dcc2dc178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:02.071784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3c269-c2b0-4d1d-9d1d-087dcc2dc178 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:02.072211 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3c269-c2b0-4d1d-9d1d-087dcc2dc178 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:02.074128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3c269-c2b0-4d1d-9d1d-087dcc2dc178 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:02.111113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916bad5-9b24-4d74-91ea-98c0ae7b1dc5 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:02.115914 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76312a5f-11b5-4557-aaeb-2bcebf6fa598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:02.116609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76312a5f-11b5-4557-aaeb-2bcebf6fa598 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:02.116994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76312a5f-11b5-4557-aaeb-2bcebf6fa598 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:02.117391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76312a5f-11b5-4557-aaeb-2bcebf6fa598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:02.128114 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76312a5f-11b5-4557-aaeb-2bcebf6fa598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:02.130247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76312a5f-11b5-4557-aaeb-2bcebf6fa598 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:02.144984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 817/1618] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:02.145969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76312a5f-11b5-4557-aaeb-2bcebf6fa598 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:02.155773 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76312a5f-11b5-4557-aaeb-2bcebf6fa598 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:02.159910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956d825-77e3-462f-aca0-9ef7cc554dd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:02.160666 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956d825-77e3-462f-aca0-9ef7cc554dd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:02.161053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956d825-77e3-462f-aca0-9ef7cc554dd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:02.161438 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956d825-77e3-462f-aca0-9ef7cc554dd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:02.171607 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956d825-77e3-462f-aca0-9ef7cc554dd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:02.173180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956d825-77e3-462f-aca0-9ef7cc554dd9 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:02.182807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 818/1619] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:02.183594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956d825-77e3-462f-aca0-9ef7cc554dd9 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:02.184020 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 802/1620] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:02 2022] POST /identity/v3/projects => generated 365 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:02.184701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3c269-c2b0-4d1d-9d1d-087dcc2dc178 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:02.194303 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956d825-77e3-462f-aca0-9ef7cc554dd9 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:02.213257 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:02.246962 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7654a-5f9b-4572-90e2-5aee2f6b6950 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:02.247533 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7654a-5f9b-4572-90e2-5aee2f6b6950 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:02.247816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7654a-5f9b-4572-90e2-5aee2f6b6950 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:02.248091 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7654a-5f9b-4572-90e2-5aee2f6b6950 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:02.251888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3c269-c2b0-4d1d-9d1d-087dcc2dc178 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:02.256655 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec727691-aa0b-45d1-8a80-257d8ce0a27d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:02.257151 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec727691-aa0b-45d1-8a80-257d8ce0a27d service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:02.257436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec727691-aa0b-45d1-8a80-257d8ce0a27d service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:02.257715 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec727691-aa0b-45d1-8a80-257d8ce0a27d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:02.325758 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb7654a-5f9b-4572-90e2-5aee2f6b6950 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:02.327144 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec727691-aa0b-45d1-8a80-257d8ce0a27d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:02.327272 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb7654a-5f9b-4572-90e2-5aee2f6b6950 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:02.328743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec727691-aa0b-45d1-8a80-257d8ce0a27d service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:02.338427 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 819/1621] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:02.339294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7654a-5f9b-4572-90e2-5aee2f6b6950 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:02.340608 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 803/1622] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:02.341986 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d322f9-2270-4bcd-971b-3bec32699b7f None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:02.342381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d322f9-2270-4bcd-971b-3bec32699b7f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:02.342976 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d322f9-2270-4bcd-971b-3bec32699b7f None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:02.344933 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 804/1623] 10.222.0.13 () {54 vars in 912 bytes} [Thu Jun 23 11:44:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:44:02.345810 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d322f9-2270-4bcd-971b-3bec32699b7f None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:02.361580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d322f9-2270-4bcd-971b-3bec32699b7f None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:02.366140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ba504-24a2-44f4-ade9-78931a9f95a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:02.366961 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ba504-24a2-44f4-ade9-78931a9f95a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:02.367347 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ba504-24a2-44f4-ade9-78931a9f95a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:02.367738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ba504-24a2-44f4-ade9-78931a9f95a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:02.368548 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ba504-24a2-44f4-ade9-78931a9f95a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:02.370652 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ba504-24a2-44f4-ade9-78931a9f95a8 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:02.454479 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 805/1624] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:02 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:02.456853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc7e77-44c9-48a4-a7f5-5305e74d0a3a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:02.459310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc7e77-44c9-48a4-a7f5-5305e74d0a3a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:02.459616 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc7e77-44c9-48a4-a7f5-5305e74d0a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:02.466448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7654a-5f9b-4572-90e2-5aee2f6b6950 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:02.474496 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7801b-0b7b-4ed1-999d-158fb52bfdca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:02.475259 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7801b-0b7b-4ed1-999d-158fb52bfdca service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:02.475545 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7801b-0b7b-4ed1-999d-158fb52bfdca service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:02.475820 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7801b-0b7b-4ed1-999d-158fb52bfdca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:02.486801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b5cc7e77-44c9-48a4-a7f5-5305e74d0a3a None None] MFA Rules not processed for user `161c5ed1e47e450290a1f428ee100b68`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:02.487907 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b5cc7e77-44c9-48a4-a7f5-5305e74d0a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:02.510107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 806/1625] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 23 11:44:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:02.511171 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc7e77-44c9-48a4-a7f5-5305e74d0a3a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:02.532737 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc7e77-44c9-48a4-a7f5-5305e74d0a3a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:02.540367 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d7166-91ce-4b2f-9df9-18ce3aff1720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:02.541259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d7166-91ce-4b2f-9df9-18ce3aff1720 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:02.541551 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d7166-91ce-4b2f-9df9-18ce3aff1720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:02.541829 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d7166-91ce-4b2f-9df9-18ce3aff1720 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:02.542765 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d7166-91ce-4b2f-9df9-18ce3aff1720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:02.547189 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d7166-91ce-4b2f-9df9-18ce3aff1720 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:02.562913 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 807/1626] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:02.564235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d7166-91ce-4b2f-9df9-18ce3aff1720 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:02.620554 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7801b-0b7b-4ed1-999d-158fb52bfdca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:02.622874 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7801b-0b7b-4ed1-999d-158fb52bfdca service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:02.639741 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:02.646034 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 820/1627] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:02.649252 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7801b-0b7b-4ed1-999d-158fb52bfdca service nova] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:02.652606 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d7166-91ce-4b2f-9df9-18ce3aff1720 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:02.660697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fccf8-32a2-4597-a9b7-eab351f6b42c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:02.661463 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fccf8-32a2-4597-a9b7-eab351f6b42c service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:02.661911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fccf8-32a2-4597-a9b7-eab351f6b42c service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:02.662294 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fccf8-32a2-4597-a9b7-eab351f6b42c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:02.663334 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7801b-0b7b-4ed1-999d-158fb52bfdca service nova] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:02.668686 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013528f5-b3f8-4757-99e5-b37b025303d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:02.669697 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013528f5-b3f8-4757-99e5-b37b025303d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:02.669988 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013528f5-b3f8-4757-99e5-b37b025303d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:02.670266 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013528f5-b3f8-4757-99e5-b37b025303d5 admin admin] PATH_INFO: `/v3/projects/8ebf158378db42608ae39baaf29589f9/users/92e9bede2f88450a881ba145c10bc739/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:02.674086 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:02.679886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57fccf8-32a2-4597-a9b7-eab351f6b42c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:02.682765 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57fccf8-32a2-4597-a9b7-eab351f6b42c service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:02.692038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013528f5-b3f8-4757-99e5-b37b025303d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ebf158378db42608ae39baaf29589f9, user_id=92e9bede2f88450a881ba145c10bc739, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:02.695285 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013528f5-b3f8-4757-99e5-b37b025303d5 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:02.700525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 808/1628] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:02.786661 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 821/1629] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:02 2022] PUT /identity/v3/projects/8ebf158378db42608ae39baaf29589f9/users/92e9bede2f88450a881ba145c10bc739/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:02.787971 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fccf8-32a2-4597-a9b7-eab351f6b42c service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:02.805372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fccf8-32a2-4597-a9b7-eab351f6b42c service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:02.898948 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1897ac-71f6-4cfe-bbe0-075522ce8ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:02.900003 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1897ac-71f6-4cfe-bbe0-075522ce8ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:02.900590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1897ac-71f6-4cfe-bbe0-075522ce8ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:02.901174 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1897ac-71f6-4cfe-bbe0-075522ce8ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:02.901985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1897ac-71f6-4cfe-bbe0-075522ce8ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:02.904600 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1897ac-71f6-4cfe-bbe0-075522ce8ec8 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:02.921856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 809/1630] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:02.923160 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013528f5-b3f8-4757-99e5-b37b025303d5 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:02.951935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013528f5-b3f8-4757-99e5-b37b025303d5 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:02.962948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf53ea2-04eb-475a-b07e-244e7ae795f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:02.963865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf53ea2-04eb-475a-b07e-244e7ae795f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:02.964486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf53ea2-04eb-475a-b07e-244e7ae795f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:02.964757 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf53ea2-04eb-475a-b07e-244e7ae795f7 admin admin] PATH_INFO: `/v3/projects/8ebf158378db42608ae39baaf29589f9/users/92e9bede2f88450a881ba145c10bc739/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:02.970974 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf53ea2-04eb-475a-b07e-244e7ae795f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ebf158378db42608ae39baaf29589f9, user_id=92e9bede2f88450a881ba145c10bc739, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:02.973643 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf53ea2-04eb-475a-b07e-244e7ae795f7 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:03.011569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 822/1631] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:02 2022] PUT /identity/v3/projects/8ebf158378db42608ae39baaf29589f9/users/92e9bede2f88450a881ba145c10bc739/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:03.027763 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b150a0-e98c-42a1-a6e4-e631748c7581 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:03.027763 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b150a0-e98c-42a1-a6e4-e631748c7581 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:03.027763 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b150a0-e98c-42a1-a6e4-e631748c7581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:03.079829 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-00b150a0-e98c-42a1-a6e4-e631748c7581 None None] MFA Rules not processed for user `92e9bede2f88450a881ba145c10bc739`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:03.079829 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-00b150a0-e98c-42a1-a6e4-e631748c7581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:03.107485 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f442d9b-b15e-498d-9e0e-f8df556c301e None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:03.107889 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f442d9b-b15e-498d-9e0e-f8df556c301e None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:03.108600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f442d9b-b15e-498d-9e0e-f8df556c301e None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:03.110559 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 823/1632] 10.222.0.13 () {54 vars in 912 bytes} [Thu Jun 23 11:44:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:44:03.116087 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf382dc-f7cd-4fd0-9b29-4631523359be None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:03.116375 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf382dc-f7cd-4fd0-9b29-4631523359be None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:03.116652 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf382dc-f7cd-4fd0-9b29-4631523359be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:03.153378 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-3bf382dc-f7cd-4fd0-9b29-4631523359be None None] MFA Rules not processed for user `161c5ed1e47e450290a1f428ee100b68`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:03.154784 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-3bf382dc-f7cd-4fd0-9b29-4631523359be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:03.229678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 810/1633] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:03 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:03.241300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b150a0-e98c-42a1-a6e4-e631748c7581 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:03.320870 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 824/1634] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 23 11:44:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:03.345255 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b150a0-e98c-42a1-a6e4-e631748c7581 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:03.350331 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b811d2-9b61-4330-9b21-d0d777215b76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:03.351088 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b811d2-9b61-4330-9b21-d0d777215b76 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:03.352606 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b811d2-9b61-4330-9b21-d0d777215b76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:03.353012 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b811d2-9b61-4330-9b21-d0d777215b76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:03.365915 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b811d2-9b61-4330-9b21-d0d777215b76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:03.367516 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b811d2-9b61-4330-9b21-d0d777215b76 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:03.381455 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 811/1635] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:03 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:03.543906 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf382dc-f7cd-4fd0-9b29-4631523359be None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:03.557451 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf382dc-f7cd-4fd0-9b29-4631523359be None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:03.565381 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cfb5e-3795-46ee-814b-a3d3f2a8c5d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:03.566444 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cfb5e-3795-46ee-814b-a3d3f2a8c5d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:03.569792 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cfb5e-3795-46ee-814b-a3d3f2a8c5d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:03.570220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cfb5e-3795-46ee-814b-a3d3f2a8c5d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:03.604921 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9cfb5e-3795-46ee-814b-a3d3f2a8c5d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:03.606670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9cfb5e-3795-46ee-814b-a3d3f2a8c5d8 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:03.621994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 825/1636] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:04.344509 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b811d2-9b61-4330-9b21-d0d777215b76 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:04.355502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b811d2-9b61-4330-9b21-d0d777215b76 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:04.367362 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37190c-0e07-4dfd-ab28-fe81c2f75090 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:04.368050 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37190c-0e07-4dfd-ab28-fe81c2f75090 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:04.368312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37190c-0e07-4dfd-ab28-fe81c2f75090 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:04.368588 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37190c-0e07-4dfd-ab28-fe81c2f75090 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:04.394092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:04.451236 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37190c-0e07-4dfd-ab28-fe81c2f75090 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:04.452760 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37190c-0e07-4dfd-ab28-fe81c2f75090 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:04.462857 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 812/1637] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:04 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:05.263424 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae271ad-ca50-4ec6-b2b1-e6e66deb0bd7 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:05.263967 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae271ad-ca50-4ec6-b2b1-e6e66deb0bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:05.263967 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae271ad-ca50-4ec6-b2b1-e6e66deb0bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:05.310551 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-dae271ad-ca50-4ec6-b2b1-e6e66deb0bd7 None None] MFA Rules not processed for user `f13429cf5e49438b987a49564c345e3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:05.312179 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-dae271ad-ca50-4ec6-b2b1-e6e66deb0bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:05.415578 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 826/1638] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:05 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:05.424055 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37190c-0e07-4dfd-ab28-fe81c2f75090 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:05.531985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37190c-0e07-4dfd-ab28-fe81c2f75090 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:05.538504 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de22b2-c294-46ba-9405-24c8c551914b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:05.540063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de22b2-c294-46ba-9405-24c8c551914b admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:05.540063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de22b2-c294-46ba-9405-24c8c551914b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:05.541176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de22b2-c294-46ba-9405-24c8c551914b admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:05.542049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de22b2-c294-46ba-9405-24c8c551914b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:05.545050 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de22b2-c294-46ba-9405-24c8c551914b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:05.701190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 813/1639] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:05 2022] POST /identity/v3/projects => generated 405 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:05.706759 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae271ad-ca50-4ec6-b2b1-e6e66deb0bd7 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:05.723498 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae271ad-ca50-4ec6-b2b1-e6e66deb0bd7 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:05.806600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:05.834271 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da74ab-1ea5-4243-a06a-8e0df0821261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:05.834961 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da74ab-1ea5-4243-a06a-8e0df0821261 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:05.835405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da74ab-1ea5-4243-a06a-8e0df0821261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:05.835796 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da74ab-1ea5-4243-a06a-8e0df0821261 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:05.836598 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da74ab-1ea5-4243-a06a-8e0df0821261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:05.838869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da74ab-1ea5-4243-a06a-8e0df0821261 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:05.930942 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 827/1640] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:05 2022] POST /identity/v3/users => generated 309 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:05.935360 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de22b2-c294-46ba-9405-24c8c551914b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:05.952246 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de22b2-c294-46ba-9405-24c8c551914b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:05.960852 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcd0e7-691e-44ef-b65a-f4c69828c201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:05.961542 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcd0e7-691e-44ef-b65a-f4c69828c201 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:05.962239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcd0e7-691e-44ef-b65a-f4c69828c201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:05.962368 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcd0e7-691e-44ef-b65a-f4c69828c201 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:05.963470 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fcd0e7-691e-44ef-b65a-f4c69828c201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:05.965098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fcd0e7-691e-44ef-b65a-f4c69828c201 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:05.982180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 814/1641] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:05.984530 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da74ab-1ea5-4243-a06a-8e0df0821261 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:05.997710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da74ab-1ea5-4243-a06a-8e0df0821261 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:06.012579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b69df6-fd74-45f1-a9e3-15390dea5a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:06.013539 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b69df6-fd74-45f1-a9e3-15390dea5a41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:06.013824 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b69df6-fd74-45f1-a9e3-15390dea5a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:06.014099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b69df6-fd74-45f1-a9e3-15390dea5a41 admin admin] PATH_INFO: `/v3/projects/34769d18070c417280c6cbf54c825d1d/users/0d0ae48475a84ed98772d347dfd5ddfb/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:06.349265 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b69df6-fd74-45f1-a9e3-15390dea5a41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34769d18070c417280c6cbf54c825d1d, user_id=0d0ae48475a84ed98772d347dfd5ddfb, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:06.349265 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b69df6-fd74-45f1-a9e3-15390dea5a41 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:06.349265 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 828/1642] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:05 2022] PUT /identity/v3/projects/34769d18070c417280c6cbf54c825d1d/users/0d0ae48475a84ed98772d347dfd5ddfb/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:06.351469 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcd0e7-691e-44ef-b65a-f4c69828c201 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:06.351469 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcd0e7-691e-44ef-b65a-f4c69828c201 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:06.360039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa50ef8-db17-4fd9-91c8-3c209b280e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:06.360858 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa50ef8-db17-4fd9-91c8-3c209b280e14 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:06.362616 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa50ef8-db17-4fd9-91c8-3c209b280e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:06.362902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa50ef8-db17-4fd9-91c8-3c209b280e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:06.363669 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa50ef8-db17-4fd9-91c8-3c209b280e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:06.369663 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa50ef8-db17-4fd9-91c8-3c209b280e14 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:06.399363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b69df6-fd74-45f1-a9e3-15390dea5a41 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:06.408603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 815/1643] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:06.412483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b69df6-fd74-45f1-a9e3-15390dea5a41 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:06.418574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61e398-0ed7-4edd-86c3-07a09e9340ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:06.419579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61e398-0ed7-4edd-86c3-07a09e9340ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:06.422861 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61e398-0ed7-4edd-86c3-07a09e9340ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:06.422861 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61e398-0ed7-4edd-86c3-07a09e9340ff admin admin] PATH_INFO: `/v3/projects/34769d18070c417280c6cbf54c825d1d/users/0d0ae48475a84ed98772d347dfd5ddfb/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:06.427474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a61e398-0ed7-4edd-86c3-07a09e9340ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=34769d18070c417280c6cbf54c825d1d, user_id=0d0ae48475a84ed98772d347dfd5ddfb, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:06.430806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a61e398-0ed7-4edd-86c3-07a09e9340ff admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:06.445788 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 829/1644] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:06 2022] PUT /identity/v3/projects/34769d18070c417280c6cbf54c825d1d/users/0d0ae48475a84ed98772d347dfd5ddfb/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:08.210670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa50ef8-db17-4fd9-91c8-3c209b280e14 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:08.239942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa50ef8-db17-4fd9-91c8-3c209b280e14 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:08.320240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:08.438188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47adf90-83d5-4ba8-962d-d537a1470568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:08.439708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47adf90-83d5-4ba8-962d-d537a1470568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:08.440053 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47adf90-83d5-4ba8-962d-d537a1470568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:08.440333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47adf90-83d5-4ba8-962d-d537a1470568 admin admin] PATH_INFO: `/v3/users/40552f9345a34e86b8a0a791bef6b7bb` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:08.448148 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47adf90-83d5-4ba8-962d-d537a1470568 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40552f9345a34e86b8a0a791bef6b7bb)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:08.449871 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: WARNING py.warnings [None req-e47adf90-83d5-4ba8-962d-d537a1470568 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:44:08.449871 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: warnings.warn(msg) Jun 23 11:44:08.449871 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: Jun 23 11:44:08.450524 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47adf90-83d5-4ba8-962d-d537a1470568 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:08.927879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e47adf90-83d5-4ba8-962d-d537a1470568 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40552f9345a34e86b8a0a791bef6b7bb'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:08.943328 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e47adf90-83d5-4ba8-962d-d537a1470568 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40552f9345a34e86b8a0a791bef6b7bb'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:08.948483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e47adf90-83d5-4ba8-962d-d537a1470568 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40552f9345a34e86b8a0a791bef6b7bb'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:08.954182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e47adf90-83d5-4ba8-962d-d537a1470568 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40552f9345a34e86b8a0a791bef6b7bb'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:09.062753 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e47adf90-83d5-4ba8-962d-d537a1470568 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40552f9345a34e86b8a0a791bef6b7bb'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:09.144098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 816/1645] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:08 2022] DELETE /identity/v3/users/40552f9345a34e86b8a0a791bef6b7bb => generated 0 bytes in 930 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:09.145679 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69216bb-bf4f-423a-ab7a-71ea0c2077f9 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:09.146253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69216bb-bf4f-423a-ab7a-71ea0c2077f9 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:09.146253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69216bb-bf4f-423a-ab7a-71ea0c2077f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:09.197346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-a69216bb-bf4f-423a-ab7a-71ea0c2077f9 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:09.199399 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a69216bb-bf4f-423a-ab7a-71ea0c2077f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:09.393512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 830/1646] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:44:09.447142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47adf90-83d5-4ba8-962d-d537a1470568 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:09.449379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e47adf90-83d5-4ba8-962d-d537a1470568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:09.657331 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47adf90-83d5-4ba8-962d-d537a1470568 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:09.661179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1558c7-6106-49b6-909c-827b61871859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:09.661850 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1558c7-6106-49b6-909c-827b61871859 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:09.662135 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1558c7-6106-49b6-909c-827b61871859 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:09.662425 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1558c7-6106-49b6-909c-827b61871859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:09.687395 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1558c7-6106-49b6-909c-827b61871859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:09.688955 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1558c7-6106-49b6-909c-827b61871859 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:09.703939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 817/1647] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:10.098861 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635938fd-f06d-4f27-ade7-47f4a3066c67 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:10.099198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635938fd-f06d-4f27-ade7-47f4a3066c67 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:10.099481 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635938fd-f06d-4f27-ade7-47f4a3066c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:10.159605 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-635938fd-f06d-4f27-ade7-47f4a3066c67 None None] MFA Rules not processed for user `0d0ae48475a84ed98772d347dfd5ddfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:10.160846 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-635938fd-f06d-4f27-ade7-47f4a3066c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:10.220090 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac2995-7733-4959-a4c0-8726593d3957 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:10.220320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac2995-7733-4959-a4c0-8726593d3957 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:10.220626 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac2995-7733-4959-a4c0-8726593d3957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:10.272295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-deac2995-7733-4959-a4c0-8726593d3957 None None] MFA Rules not processed for user `b7fd8323d29d445f903e2103453e8ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:10.273552 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-deac2995-7733-4959-a4c0-8726593d3957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:10.326418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 831/1648] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:10 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:10.362229 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:10.437936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 818/1649] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:10.453241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635938fd-f06d-4f27-ade7-47f4a3066c67 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:10.455183 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-635938fd-f06d-4f27-ade7-47f4a3066c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:10.525778 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:10.653597 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635938fd-f06d-4f27-ade7-47f4a3066c67 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:10.662016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de0fa5-2eb5-4547-bcca-b0f9281bfa9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:10.662929 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de0fa5-2eb5-4547-bcca-b0f9281bfa9f service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:10.663401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de0fa5-2eb5-4547-bcca-b0f9281bfa9f service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:10.663852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de0fa5-2eb5-4547-bcca-b0f9281bfa9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:10.673769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de0fa5-2eb5-4547-bcca-b0f9281bfa9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:10.675854 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de0fa5-2eb5-4547-bcca-b0f9281bfa9f service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:10.689226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 832/1650] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:11.002058 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac2995-7733-4959-a4c0-8726593d3957 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:11.003615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-deac2995-7733-4959-a4c0-8726593d3957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:11.025720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac2995-7733-4959-a4c0-8726593d3957 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:11.036749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfc5fb-971c-45cd-84b7-4f86b66dbc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:11.037642 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfc5fb-971c-45cd-84b7-4f86b66dbc1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:11.039998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfc5fb-971c-45cd-84b7-4f86b66dbc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:11.043282 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfc5fb-971c-45cd-84b7-4f86b66dbc1f admin admin] PATH_INFO: `/v3/projects/0252f30c9ada49a081547315cb6dcc22` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:11.044731 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfc5fb-971c-45cd-84b7-4f86b66dbc1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0252f30c9ada49a081547315cb6dcc22)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:11.048885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfc5fb-971c-45cd-84b7-4f86b66dbc1f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:11.238219 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-febfc5fb-971c-45cd-84b7-4f86b66dbc1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0252f30c9ada49a081547315cb6dcc22', 'request_id': 'req-febfc5fb-971c-45cd-84b7-4f86b66dbc1f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:11.250027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-febfc5fb-971c-45cd-84b7-4f86b66dbc1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0252f30c9ada49a081547315cb6dcc22', 'request_id': 'req-febfc5fb-971c-45cd-84b7-4f86b66dbc1f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:11.336575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-febfc5fb-971c-45cd-84b7-4f86b66dbc1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0252f30c9ada49a081547315cb6dcc22', 'request_id': 'req-febfc5fb-971c-45cd-84b7-4f86b66dbc1f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:11.352512 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-febfc5fb-971c-45cd-84b7-4f86b66dbc1f admin admin] The token cache is being invalidate because project 0252f30c9ada49a081547315cb6dcc22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:11.353026 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-febfc5fb-971c-45cd-84b7-4f86b66dbc1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:11.355567 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-febfc5fb-971c-45cd-84b7-4f86b66dbc1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:11.358204 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 819/1651] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:11 2022] DELETE /identity/v3/projects/0252f30c9ada49a081547315cb6dcc22 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:11.363533 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19697f7e-43fb-4f2a-bdfc-8350f2cd2316 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:11.368582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19697f7e-43fb-4f2a-bdfc-8350f2cd2316 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:11.368582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19697f7e-43fb-4f2a-bdfc-8350f2cd2316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:11.387633 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-19697f7e-43fb-4f2a-bdfc-8350f2cd2316 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:11.388888 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-19697f7e-43fb-4f2a-bdfc-8350f2cd2316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:11.517309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ed3e0-c0ca-471a-91cc-8fbdef51dc4e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:11.517599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ed3e0-c0ca-471a-91cc-8fbdef51dc4e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:11.517872 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ed3e0-c0ca-471a-91cc-8fbdef51dc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:11.519473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 833/1652] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:11.554039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ce8ed3e0-c0ca-471a-91cc-8fbdef51dc4e None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:11.555319 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ce8ed3e0-c0ca-471a-91cc-8fbdef51dc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:11.574889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 820/1653] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:11.577771 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19697f7e-43fb-4f2a-bdfc-8350f2cd2316 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:11.589341 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19697f7e-43fb-4f2a-bdfc-8350f2cd2316 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:11.595978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c16816-506b-429c-bf05-21ca56057c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:11.596697 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c16816-506b-429c-bf05-21ca56057c9b admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:11.597030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c16816-506b-429c-bf05-21ca56057c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:11.597250 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c16816-506b-429c-bf05-21ca56057c9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:11.598228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c16816-506b-429c-bf05-21ca56057c9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:11.598557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c16816-506b-429c-bf05-21ca56057c9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:11.600690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c16816-506b-429c-bf05-21ca56057c9b admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:11.628390 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ed3e0-c0ca-471a-91cc-8fbdef51dc4e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:11.650403 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 834/1654] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:44:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:44:11.651996 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ed3e0-c0ca-471a-91cc-8fbdef51dc4e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:11.666079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373a354-53ce-4a22-82fc-c6905a8cfd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:11.666840 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373a354-53ce-4a22-82fc-c6905a8cfd96 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:11.667122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373a354-53ce-4a22-82fc-c6905a8cfd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:11.667393 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373a354-53ce-4a22-82fc-c6905a8cfd96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:11.668097 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373a354-53ce-4a22-82fc-c6905a8cfd96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:11.670044 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373a354-53ce-4a22-82fc-c6905a8cfd96 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:11.767030 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 821/1655] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:11 2022] POST /identity/v3/projects => generated 367 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:11.772938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c16816-506b-429c-bf05-21ca56057c9b admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:11.801260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c16816-506b-429c-bf05-21ca56057c9b admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:11.936411 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce0012-963f-4cf7-88d0-a3f48395b679 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:11.936908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce0012-963f-4cf7-88d0-a3f48395b679 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:11.941001 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce0012-963f-4cf7-88d0-a3f48395b679 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:11.941001 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 822/1656] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 23 11:44:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:44:11.982105 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b402fa-307e-46be-b128-6d9a9b89f16e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:11.982397 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b402fa-307e-46be-b128-6d9a9b89f16e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:11.982705 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b402fa-307e-46be-b128-6d9a9b89f16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:12.022826 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc6123-49e2-43ab-9312-ed1f79027ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:12.023854 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc6123-49e2-43ab-9312-ed1f79027ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:12.024402 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc6123-49e2-43ab-9312-ed1f79027ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:12.024994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc6123-49e2-43ab-9312-ed1f79027ed1 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:12.030849 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc6123-49e2-43ab-9312-ed1f79027ed1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:12.031017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-50b402fa-307e-46be-b128-6d9a9b89f16e None None] MFA Rules not processed for user `ac04b32777ca4f929e9e59deee6e0260`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:12.033917 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-50b402fa-307e-46be-b128-6d9a9b89f16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:12.034766 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc6123-49e2-43ab-9312-ed1f79027ed1 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:12.147747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 823/1657] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 23 11:44:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:12.188081 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b402fa-307e-46be-b128-6d9a9b89f16e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:12.190156 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-50b402fa-307e-46be-b128-6d9a9b89f16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:12.192195 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 835/1658] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:11 2022] POST /identity/v3/users => generated 284 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:12.198100 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc6123-49e2-43ab-9312-ed1f79027ed1 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:12.219735 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b402fa-307e-46be-b128-6d9a9b89f16e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:12.220831 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc6123-49e2-43ab-9312-ed1f79027ed1 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:12.229527 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0db18-fdc5-4e11-9ab0-578e976ba6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:12.231507 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629419f-ef7a-4cb5-9cc8-4b4106051710 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac04b32777ca4f929e9e59deee6e0260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:12.232326 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629419f-ef7a-4cb5-9cc8-4b4106051710 service placement] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:12.233730 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0db18-fdc5-4e11-9ab0-578e976ba6ac admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:12.234260 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629419f-ef7a-4cb5-9cc8-4b4106051710 service placement] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:12.234717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629419f-ef7a-4cb5-9cc8-4b4106051710 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:12.235659 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0db18-fdc5-4e11-9ab0-578e976ba6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:12.237184 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0db18-fdc5-4e11-9ab0-578e976ba6ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:12.239459 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0db18-fdc5-4e11-9ab0-578e976ba6ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:12.241986 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0db18-fdc5-4e11-9ab0-578e976ba6ac admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:12.254070 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629419f-ef7a-4cb5-9cc8-4b4106051710 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:12.256040 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629419f-ef7a-4cb5-9cc8-4b4106051710 service placement] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:12.263764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 836/1659] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:12.267004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0db18-fdc5-4e11-9ab0-578e976ba6ac admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:12.277322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 824/1660] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:12.283685 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0db18-fdc5-4e11-9ab0-578e976ba6ac admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:12.288974 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16538e-d440-428e-b169-ce0daa4ded76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:12.291564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16538e-d440-428e-b169-ce0daa4ded76 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:12.292041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16538e-d440-428e-b169-ce0daa4ded76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:12.292457 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16538e-d440-428e-b169-ce0daa4ded76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:12.293355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16538e-d440-428e-b169-ce0daa4ded76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:12.295157 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16538e-d440-428e-b169-ce0daa4ded76 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:12.297677 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:12.315520 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629419f-ef7a-4cb5-9cc8-4b4106051710 service placement] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:12.321009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 837/1661] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:12.327798 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629419f-ef7a-4cb5-9cc8-4b4106051710 service placement] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:12.333024 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914899c-f108-431b-adce-c5980f8f5f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:12.334381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914899c-f108-431b-adce-c5980f8f5f14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:12.334986 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914899c-f108-431b-adce-c5980f8f5f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:12.335421 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914899c-f108-431b-adce-c5980f8f5f14 admin admin] PATH_INFO: `/v3/projects/cda6be175c8f45b196c36e6b89aa6d1e/users/4f5d9af6661c475b91f0da76e938c567/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:12.341236 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:12.506211 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914899c-f108-431b-adce-c5980f8f5f14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cda6be175c8f45b196c36e6b89aa6d1e, user_id=4f5d9af6661c475b91f0da76e938c567, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:12.509957 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914899c-f108-431b-adce-c5980f8f5f14 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:12.650495 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 825/1662] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:12 2022] PUT /identity/v3/projects/cda6be175c8f45b196c36e6b89aa6d1e/users/4f5d9af6661c475b91f0da76e938c567/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:12.665006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e666209-c48e-4a47-9214-fd73e82e0f48 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:12.665814 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e666209-c48e-4a47-9214-fd73e82e0f48 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:12.667036 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e666209-c48e-4a47-9214-fd73e82e0f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:12.779223 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-1e666209-c48e-4a47-9214-fd73e82e0f48 None None] MFA Rules not processed for user `4f5d9af6661c475b91f0da76e938c567`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:12.791291 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-1e666209-c48e-4a47-9214-fd73e82e0f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:12.906382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914899c-f108-431b-adce-c5980f8f5f14 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:12.906836 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 838/1663] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:12.908163 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1914899c-f108-431b-adce-c5980f8f5f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:13.007384 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914899c-f108-431b-adce-c5980f8f5f14 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:13.012646 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347d264-466b-40b8-af48-a8357b7fb223 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:13.014831 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347d264-466b-40b8-af48-a8357b7fb223 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:13.015203 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347d264-466b-40b8-af48-a8357b7fb223 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:13.015544 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347d264-466b-40b8-af48-a8357b7fb223 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:13.028717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a347d264-466b-40b8-af48-a8357b7fb223 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:13.030950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a347d264-466b-40b8-af48-a8357b7fb223 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:13.064494 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 826/1664] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:12 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:14.856863 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e666209-c48e-4a47-9214-fd73e82e0f48 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:14.859139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-1e666209-c48e-4a47-9214-fd73e82e0f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:15.008282 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e666209-c48e-4a47-9214-fd73e82e0f48 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:15.023985 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031af652-2028-481a-a718-6bca31b938e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:15.025561 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031af652-2028-481a-a718-6bca31b938e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:15.025949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031af652-2028-481a-a718-6bca31b938e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:15.026566 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031af652-2028-481a-a718-6bca31b938e1 admin admin] PATH_INFO: `/v3/users/f13429cf5e49438b987a49564c345e3d` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:15.029401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031af652-2028-481a-a718-6bca31b938e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f13429cf5e49438b987a49564c345e3d)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:15.033807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031af652-2028-481a-a718-6bca31b938e1 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:15.313695 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-031af652-2028-481a-a718-6bca31b938e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f13429cf5e49438b987a49564c345e3d'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:15.340384 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-031af652-2028-481a-a718-6bca31b938e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f13429cf5e49438b987a49564c345e3d'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:15.343948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-031af652-2028-481a-a718-6bca31b938e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f13429cf5e49438b987a49564c345e3d'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:15.344823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-031af652-2028-481a-a718-6bca31b938e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f13429cf5e49438b987a49564c345e3d'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:15.392592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-031af652-2028-481a-a718-6bca31b938e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f13429cf5e49438b987a49564c345e3d'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:15.506336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 839/1665] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:14 2022] DELETE /identity/v3/users/f13429cf5e49438b987a49564c345e3d => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:16.443087 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d602d-1882-44b7-b04b-fa91628600eb None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:16.443574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d602d-1882-44b7-b04b-fa91628600eb None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:16.443795 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d602d-1882-44b7-b04b-fa91628600eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:16.508881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-599d602d-1882-44b7-b04b-fa91628600eb None None] MFA Rules not processed for user `bcd5fc6167654972868941ed28c327b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:16.510431 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-599d602d-1882-44b7-b04b-fa91628600eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:16.529937 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031af652-2028-481a-a718-6bca31b938e1 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:16.529937 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-031af652-2028-481a-a718-6bca31b938e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:16.536844 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:16.768460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 827/1666] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:16.783131 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dabfc-8fb2-43ea-b855-ed0861322b98 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:16.783416 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dabfc-8fb2-43ea-b855-ed0861322b98 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:16.783720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dabfc-8fb2-43ea-b855-ed0861322b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:16.789546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031af652-2028-481a-a718-6bca31b938e1 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:16.820260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa3a96-ee61-4d66-92a4-880b95b8e963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:16.821145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa3a96-ee61-4d66-92a4-880b95b8e963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:16.821503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa3a96-ee61-4d66-92a4-880b95b8e963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:16.821682 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa3a96-ee61-4d66-92a4-880b95b8e963 admin admin] PATH_INFO: `/v3/projects/e33164546fd046309332f7e8b447dfb0` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:16.831941 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa3a96-ee61-4d66-92a4-880b95b8e963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e33164546fd046309332f7e8b447dfb0)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:16.831941 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: WARNING py.warnings [None req-30fa3a96-ee61-4d66-92a4-880b95b8e963 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:44:16.831941 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: warnings.warn(msg) Jun 23 11:44:16.831941 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: Jun 23 11:44:16.832761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa3a96-ee61-4d66-92a4-880b95b8e963 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:16.844617 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c91dabfc-8fb2-43ea-b855-ed0861322b98 None None] MFA Rules not processed for user `bcd5fc6167654972868941ed28c327b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:16.845844 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c91dabfc-8fb2-43ea-b855-ed0861322b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:16.886373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 828/1667] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:16.898847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91dabfc-8fb2-43ea-b855-ed0861322b98 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:16.901544 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c91dabfc-8fb2-43ea-b855-ed0861322b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:17.098767 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-30fa3a96-ee61-4d66-92a4-880b95b8e963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e33164546fd046309332f7e8b447dfb0', 'request_id': 'req-30fa3a96-ee61-4d66-92a4-880b95b8e963'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:17.132643 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-30fa3a96-ee61-4d66-92a4-880b95b8e963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e33164546fd046309332f7e8b447dfb0', 'request_id': 'req-30fa3a96-ee61-4d66-92a4-880b95b8e963'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:17.194848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91dabfc-8fb2-43ea-b855-ed0861322b98 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:17.208537 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:17.236978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-30fa3a96-ee61-4d66-92a4-880b95b8e963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e33164546fd046309332f7e8b447dfb0', 'request_id': 'req-30fa3a96-ee61-4d66-92a4-880b95b8e963'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:17.254078 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-30fa3a96-ee61-4d66-92a4-880b95b8e963 admin admin] The token cache is being invalidate because project e33164546fd046309332f7e8b447dfb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:17.254492 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-30fa3a96-ee61-4d66-92a4-880b95b8e963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:17.255684 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-30fa3a96-ee61-4d66-92a4-880b95b8e963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:17.259801 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 840/1668] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:16 2022] DELETE /identity/v3/projects/e33164546fd046309332f7e8b447dfb0 => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:17.263680 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa3a96-ee61-4d66-92a4-880b95b8e963 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:17.266338 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-30fa3a96-ee61-4d66-92a4-880b95b8e963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:17.360135 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd251e9-d075-4d52-91f7-14b089b9c207 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:17.361171 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd251e9-d075-4d52-91f7-14b089b9c207 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:17.361633 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd251e9-d075-4d52-91f7-14b089b9c207 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:17.362086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd251e9-d075-4d52-91f7-14b089b9c207 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:17.366506 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9bd251e9-d075-4d52-91f7-14b089b9c207 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:17.403860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa3a96-ee61-4d66-92a4-880b95b8e963 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:17.416550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcb3f9-d5f6-4ae4-9e0d-69b9413d76a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:17.417451 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcb3f9-d5f6-4ae4-9e0d-69b9413d76a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:17.418276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcb3f9-d5f6-4ae4-9e0d-69b9413d76a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:17.418276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcb3f9-d5f6-4ae4-9e0d-69b9413d76a3 admin admin] PATH_INFO: `/v3/users/0d0ae48475a84ed98772d347dfd5ddfb` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:17.419579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bcb3f9-d5f6-4ae4-9e0d-69b9413d76a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d0ae48475a84ed98772d347dfd5ddfb)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:17.425533 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bcb3f9-d5f6-4ae4-9e0d-69b9413d76a3 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:17.456607 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd251e9-d075-4d52-91f7-14b089b9c207 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:17.460751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd251e9-d075-4d52-91f7-14b089b9c207 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:17.511407 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 829/1669] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:17.769947 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:17.789354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-a7bcb3f9-d5f6-4ae4-9e0d-69b9413d76a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d0ae48475a84ed98772d347dfd5ddfb'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:17.810185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-a7bcb3f9-d5f6-4ae4-9e0d-69b9413d76a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d0ae48475a84ed98772d347dfd5ddfb'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:17.811332 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-a7bcb3f9-d5f6-4ae4-9e0d-69b9413d76a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d0ae48475a84ed98772d347dfd5ddfb'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:17.812428 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-a7bcb3f9-d5f6-4ae4-9e0d-69b9413d76a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d0ae48475a84ed98772d347dfd5ddfb'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:17.850841 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-a7bcb3f9-d5f6-4ae4-9e0d-69b9413d76a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d0ae48475a84ed98772d347dfd5ddfb'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:17.936860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 841/1670] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:17 2022] DELETE /identity/v3/users/0d0ae48475a84ed98772d347dfd5ddfb => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:18.095322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc766c45-555b-42ce-b0c8-19af456d1634 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:18.095853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc766c45-555b-42ce-b0c8-19af456d1634 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:18.096211 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc766c45-555b-42ce-b0c8-19af456d1634 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:18.098261 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 830/1671] 10.222.0.13 () {54 vars in 912 bytes} [Thu Jun 23 11:44:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:44:18.103899 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e62f0-2b19-4a03-9c41-4ba39d011147 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:18.104195 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e62f0-2b19-4a03-9c41-4ba39d011147 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:18.107678 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e62f0-2b19-4a03-9c41-4ba39d011147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:18.148108 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-2b4e62f0-2b19-4a03-9c41-4ba39d011147 None None] MFA Rules not processed for user `810fa3f54cd04f989252765f7e76e602`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:18.149361 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-2b4e62f0-2b19-4a03-9c41-4ba39d011147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:18.243466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 842/1672] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 23 11:44:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:18.272076 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc766c45-555b-42ce-b0c8-19af456d1634 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:18.274613 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bc766c45-555b-42ce-b0c8-19af456d1634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:18.299321 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc766c45-555b-42ce-b0c8-19af456d1634 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:18.303981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8e3e9-1ef4-42ee-8e64-4b5da7eeb5b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:18.304986 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8e3e9-1ef4-42ee-8e64-4b5da7eeb5b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:18.305267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8e3e9-1ef4-42ee-8e64-4b5da7eeb5b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:18.305739 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8e3e9-1ef4-42ee-8e64-4b5da7eeb5b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:18.316604 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc8e3e9-1ef4-42ee-8e64-4b5da7eeb5b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:18.318039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc8e3e9-1ef4-42ee-8e64-4b5da7eeb5b5 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:18.336019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 831/1673] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:18.788976 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e62f0-2b19-4a03-9c41-4ba39d011147 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:18.791525 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-2b4e62f0-2b19-4a03-9c41-4ba39d011147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:18.938383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e62f0-2b19-4a03-9c41-4ba39d011147 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:18.943338 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3bfbc-19c2-4731-a392-4e53fc636f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:18.944346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3bfbc-19c2-4731-a392-4e53fc636f5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:18.945501 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3bfbc-19c2-4731-a392-4e53fc636f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:18.945889 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3bfbc-19c2-4731-a392-4e53fc636f5f admin admin] PATH_INFO: `/v3/projects/34769d18070c417280c6cbf54c825d1d` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:18.947445 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3bfbc-19c2-4731-a392-4e53fc636f5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=34769d18070c417280c6cbf54c825d1d)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:18.949566 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3bfbc-19c2-4731-a392-4e53fc636f5f admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:19.151751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fd13f-48aa-4d08-8e46-e851c57d3daf None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:19.152595 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fd13f-48aa-4d08-8e46-e851c57d3daf None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:19.153038 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fd13f-48aa-4d08-8e46-e851c57d3daf None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:19.157466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c9b3bfbc-19c2-4731-a392-4e53fc636f5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34769d18070c417280c6cbf54c825d1d', 'request_id': 'req-c9b3bfbc-19c2-4731-a392-4e53fc636f5f'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:19.157873 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 832/1674] 10.222.0.13 () {54 vars in 960 bytes} [Thu Jun 23 11:44:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:44:19.164274 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:19.166732 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faed26a-3b6d-45b1-8720-846b6eca13ca None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:19.167160 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c9b3bfbc-19c2-4731-a392-4e53fc636f5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34769d18070c417280c6cbf54c825d1d', 'request_id': 'req-c9b3bfbc-19c2-4731-a392-4e53fc636f5f'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:19.180340 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faed26a-3b6d-45b1-8720-846b6eca13ca None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:19.180710 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faed26a-3b6d-45b1-8720-846b6eca13ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:19.212483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5faed26a-3b6d-45b1-8720-846b6eca13ca None None] MFA Rules not processed for user `ac04b32777ca4f929e9e59deee6e0260`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:19.214728 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5faed26a-3b6d-45b1-8720-846b6eca13ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:19.256886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c9b3bfbc-19c2-4731-a392-4e53fc636f5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34769d18070c417280c6cbf54c825d1d', 'request_id': 'req-c9b3bfbc-19c2-4731-a392-4e53fc636f5f'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:19.284476 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c9b3bfbc-19c2-4731-a392-4e53fc636f5f admin admin] The token cache is being invalidate because project 34769d18070c417280c6cbf54c825d1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:19.284476 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c9b3bfbc-19c2-4731-a392-4e53fc636f5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:19.284476 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c9b3bfbc-19c2-4731-a392-4e53fc636f5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:19.286043 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 843/1675] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:18 2022] DELETE /identity/v3/projects/34769d18070c417280c6cbf54c825d1d => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:19.288631 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8f734-6b72-4234-a927-20e797b4c84f None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:19.289454 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8f734-6b72-4234-a927-20e797b4c84f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:19.289941 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8f734-6b72-4234-a927-20e797b4c84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:19.325423 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-eaf8f734-6b72-4234-a927-20e797b4c84f None None] MFA Rules not processed for user `bcd5fc6167654972868941ed28c327b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:19.327883 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-eaf8f734-6b72-4234-a927-20e797b4c84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:19.480040 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 833/1676] 10.222.0.13 () {58 vars in 1057 bytes} [Thu Jun 23 11:44:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:19.485907 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8dd49-765d-47fc-b2f3-86d9c7eae394 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:19.486209 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8dd49-765d-47fc-b2f3-86d9c7eae394 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:19.486496 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8dd49-765d-47fc-b2f3-86d9c7eae394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:19.495928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 844/1677] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:19.517736 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8f734-6b72-4234-a927-20e797b4c84f None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:19.554687 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8f734-6b72-4234-a927-20e797b4c84f None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:19.567888 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f400402-49ed-437e-abec-a2f03798c371 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac04b32777ca4f929e9e59deee6e0260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:19.567888 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f400402-49ed-437e-abec-a2f03798c371 service placement] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:19.567888 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f400402-49ed-437e-abec-a2f03798c371 service placement] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:19.572751 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f400402-49ed-437e-abec-a2f03798c371 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:19.575173 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-7f400402-49ed-437e-abec-a2f03798c371 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:19.590018 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-61a8dd49-765d-47fc-b2f3-86d9c7eae394 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:19.590018 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-61a8dd49-765d-47fc-b2f3-86d9c7eae394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:19.607030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f400402-49ed-437e-abec-a2f03798c371 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:19.609867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f400402-49ed-437e-abec-a2f03798c371 service placement] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:19.659339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 845/1678] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:19.669488 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497feb43-be01-495f-8bf3-4fa7c3dd2bd0 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:19.669783 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497feb43-be01-495f-8bf3-4fa7c3dd2bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:19.684099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497feb43-be01-495f-8bf3-4fa7c3dd2bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:19.743101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:19.751766 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-497feb43-be01-495f-8bf3-4fa7c3dd2bd0 None None] MFA Rules not processed for user `b7fd8323d29d445f903e2103453e8ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:19.765019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-497feb43-be01-495f-8bf3-4fa7c3dd2bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:19.830889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 834/1679] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:19.833867 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23a1f3-7a5e-440c-a8f1-dc34b0a9be16 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:19.834408 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23a1f3-7a5e-440c-a8f1-dc34b0a9be16 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:19.835215 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23a1f3-7a5e-440c-a8f1-dc34b0a9be16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:19.866993 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-bc23a1f3-7a5e-440c-a8f1-dc34b0a9be16 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:19.868430 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bc23a1f3-7a5e-440c-a8f1-dc34b0a9be16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:19.893926 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 835/1680] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:19.895688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23a1f3-7a5e-440c-a8f1-dc34b0a9be16 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:19.916818 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23a1f3-7a5e-440c-a8f1-dc34b0a9be16 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:19.932225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ffa81-79a3-4b9d-b54b-9b4bb429a2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:19.932913 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ffa81-79a3-4b9d-b54b-9b4bb429a2b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:19.933308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ffa81-79a3-4b9d-b54b-9b4bb429a2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:19.933453 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ffa81-79a3-4b9d-b54b-9b4bb429a2b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:19.934188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ffa81-79a3-4b9d-b54b-9b4bb429a2b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:19.934496 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ffa81-79a3-4b9d-b54b-9b4bb429a2b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:19.936177 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ffa81-79a3-4b9d-b54b-9b4bb429a2b6 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:19.990049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 836/1681] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:44:19.991567 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ffa81-79a3-4b9d-b54b-9b4bb429a2b6 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:19.996991 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 846/1682] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:20.017648 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ffa81-79a3-4b9d-b54b-9b4bb429a2b6 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:20.030907 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ff791-1866-44cd-a61f-03b96d14db87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:20.032703 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ff791-1866-44cd-a61f-03b96d14db87 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:20.033817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ff791-1866-44cd-a61f-03b96d14db87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:20.034411 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ff791-1866-44cd-a61f-03b96d14db87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:20.035330 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ff791-1866-44cd-a61f-03b96d14db87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:20.038418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ff791-1866-44cd-a61f-03b96d14db87 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:20.197481 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc385ac1-21b9-4157-8ca8-87989d0c633c None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:20.198065 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc385ac1-21b9-4157-8ca8-87989d0c633c None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:20.198490 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc385ac1-21b9-4157-8ca8-87989d0c633c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:20.207672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 837/1683] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:19 2022] POST /identity/v3/projects => generated 381 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:20.210673 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ff791-1866-44cd-a61f-03b96d14db87 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:20.256033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-bc385ac1-21b9-4157-8ca8-87989d0c633c None None] MFA Rules not processed for user `d1b988f3241a46308b7c3683e2943b78`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:20.257275 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-bc385ac1-21b9-4157-8ca8-87989d0c633c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:20.430905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ff791-1866-44cd-a61f-03b96d14db87 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:20.443172 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2ab95-5308-48ae-b976-0b6956384814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:20.445391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2ab95-5308-48ae-b976-0b6956384814 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:20.446070 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2ab95-5308-48ae-b976-0b6956384814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:20.446070 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2ab95-5308-48ae-b976-0b6956384814 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:20.451219 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2ab95-5308-48ae-b976-0b6956384814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:20.453440 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 847/1684] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:44:20.455046 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2de49d-027b-43a3-bfc7-e5e9bc951200 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:20.455386 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2de49d-027b-43a3-bfc7-e5e9bc951200 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:20.457772 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2de49d-027b-43a3-bfc7-e5e9bc951200 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:20.458183 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2ab95-5308-48ae-b976-0b6956384814 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:20.465368 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 848/1685] 10.222.0.13 () {54 vars in 912 bytes} [Thu Jun 23 11:44:20 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:44:20.473797 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1e477-7213-4d3b-a6b9-bfcc4f94c50b None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:20.474091 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1e477-7213-4d3b-a6b9-bfcc4f94c50b None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:20.476219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1e477-7213-4d3b-a6b9-bfcc4f94c50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:20.503897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:20.519806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-dba1e477-7213-4d3b-a6b9-bfcc4f94c50b None None] MFA Rules not processed for user `810fa3f54cd04f989252765f7e76e602`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:20.521963 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-dba1e477-7213-4d3b-a6b9-bfcc4f94c50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:20.610598 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 838/1686] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:20 2022] POST /identity/v3/users => generated 291 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:20.612698 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3685b92-acc5-4557-ac89-17f3e366e8d6 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:20.613172 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3685b92-acc5-4557-ac89-17f3e366e8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:20.613800 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3685b92-acc5-4557-ac89-17f3e366e8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:20.719520 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 849/1687] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 23 11:44:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:20.720661 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1e477-7213-4d3b-a6b9-bfcc4f94c50b None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:20.733102 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1e477-7213-4d3b-a6b9-bfcc4f94c50b None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:20.737523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a3685b92-acc5-4557-ac89-17f3e366e8d6 None None] MFA Rules not processed for user `ae72d6c494ba4f16989adfe090803d47`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:20.740658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141434d-b94e-480b-bcac-cd21e0aed653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:20.740838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141434d-b94e-480b-bcac-cd21e0aed653 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:20.740976 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141434d-b94e-480b-bcac-cd21e0aed653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:20.741250 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141434d-b94e-480b-bcac-cd21e0aed653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:20.742176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141434d-b94e-480b-bcac-cd21e0aed653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:20.743360 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a3685b92-acc5-4557-ac89-17f3e366e8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:20.744551 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141434d-b94e-480b-bcac-cd21e0aed653 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:20.760547 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 850/1688] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:20.766142 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141434d-b94e-480b-bcac-cd21e0aed653 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:20.811673 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141434d-b94e-480b-bcac-cd21e0aed653 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:20.831568 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de999b0-cd1b-40eb-95c9-b689e906a56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:20.833061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de999b0-cd1b-40eb-95c9-b689e906a56c admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:20.837352 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de999b0-cd1b-40eb-95c9-b689e906a56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:20.838138 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de999b0-cd1b-40eb-95c9-b689e906a56c admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:20.840061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de999b0-cd1b-40eb-95c9-b689e906a56c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:20.845340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de999b0-cd1b-40eb-95c9-b689e906a56c admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:20.878613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 851/1689] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:20.882557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de999b0-cd1b-40eb-95c9-b689e906a56c admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:20.884526 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-6de999b0-cd1b-40eb-95c9-b689e906a56c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:20.902032 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de999b0-cd1b-40eb-95c9-b689e906a56c admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:20.906269 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51b0dc-4dc4-4b4e-b7d6-80afc7dd7e93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:20.907127 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51b0dc-4dc4-4b4e-b7d6-80afc7dd7e93 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:20.907573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51b0dc-4dc4-4b4e-b7d6-80afc7dd7e93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:20.908008 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51b0dc-4dc4-4b4e-b7d6-80afc7dd7e93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:20.921107 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51b0dc-4dc4-4b4e-b7d6-80afc7dd7e93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:20.922935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51b0dc-4dc4-4b4e-b7d6-80afc7dd7e93 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:20.927298 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 839/1690] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:20.928343 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3685b92-acc5-4557-ac89-17f3e366e8d6 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:20.933873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 852/1691] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:20.935203 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f5c00-5041-4e26-9a48-1484acf9d440 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:20.935491 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f5c00-5041-4e26-9a48-1484acf9d440 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:20.935894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f5c00-5041-4e26-9a48-1484acf9d440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:20.946063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3685b92-acc5-4557-ac89-17f3e366e8d6 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:20.962612 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3cbf1-93bc-4a1a-8689-d77010e4b800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:20.963935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3cbf1-93bc-4a1a-8689-d77010e4b800 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:20.964906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3cbf1-93bc-4a1a-8689-d77010e4b800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:20.965395 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3cbf1-93bc-4a1a-8689-d77010e4b800 admin admin] PATH_INFO: `/v3/projects/df2920638b244cd2be4b797df67f8291/users/c5b332d141e54132a169dcd755032925/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:20.971943 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-882f5c00-5041-4e26-9a48-1484acf9d440 None None] MFA Rules not processed for user `ae72d6c494ba4f16989adfe090803d47`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:20.973665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-882f5c00-5041-4e26-9a48-1484acf9d440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:20.975455 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:20.984803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 853/1692] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:20.995301 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3cbf1-93bc-4a1a-8689-d77010e4b800 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df2920638b244cd2be4b797df67f8291, user_id=c5b332d141e54132a169dcd755032925, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:21.011204 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3cbf1-93bc-4a1a-8689-d77010e4b800 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:21.141771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 840/1693] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:20 2022] PUT /identity/v3/projects/df2920638b244cd2be4b797df67f8291/users/c5b332d141e54132a169dcd755032925/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:21.142084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e41169-a8f3-4513-86a1-98eaba5b139c None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:21.142930 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e41169-a8f3-4513-86a1-98eaba5b139c None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:21.143361 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e41169-a8f3-4513-86a1-98eaba5b139c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:21.191536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-55e41169-a8f3-4513-86a1-98eaba5b139c None None] MFA Rules not processed for user `c5b332d141e54132a169dcd755032925`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:21.196319 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-55e41169-a8f3-4513-86a1-98eaba5b139c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:21.320109 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 854/1694] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:21.324292 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3cbf1-93bc-4a1a-8689-d77010e4b800 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:21.440258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3cbf1-93bc-4a1a-8689-d77010e4b800 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:21.461101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4766cbf-1148-4da7-9dd2-44db5399ccab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:21.468424 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4766cbf-1148-4da7-9dd2-44db5399ccab admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:21.468424 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4766cbf-1148-4da7-9dd2-44db5399ccab admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:21.468424 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4766cbf-1148-4da7-9dd2-44db5399ccab admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:21.468424 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4766cbf-1148-4da7-9dd2-44db5399ccab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:21.477201 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4766cbf-1148-4da7-9dd2-44db5399ccab admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:21.696897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 841/1695] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:21 2022] POST /identity/v3/projects => generated 383 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:21.700714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e41169-a8f3-4513-86a1-98eaba5b139c None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:21.724420 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e41169-a8f3-4513-86a1-98eaba5b139c None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:21.762810 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4766cbf-1148-4da7-9dd2-44db5399ccab admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:21.765012 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e4766cbf-1148-4da7-9dd2-44db5399ccab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:21.898931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1d07f-8200-44f8-a5b3-5018d1ffe0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:21.900036 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1d07f-8200-44f8-a5b3-5018d1ffe0a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:21.901617 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1d07f-8200-44f8-a5b3-5018d1ffe0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:21.902289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1d07f-8200-44f8-a5b3-5018d1ffe0a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:21.903931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1d07f-8200-44f8-a5b3-5018d1ffe0a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:21.908302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1d07f-8200-44f8-a5b3-5018d1ffe0a5 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:21.939100 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:22.034843 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 855/1696] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:21 2022] POST /identity/v3/users => generated 298 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:22.036088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1d07f-8200-44f8-a5b3-5018d1ffe0a5 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:22.043815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-56c1d07f-8200-44f8-a5b3-5018d1ffe0a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:22.086660 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4766cbf-1148-4da7-9dd2-44db5399ccab admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:22.098186 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872094e-cad0-4015-b644-37fcc5d8d89e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:22.100410 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872094e-cad0-4015-b644-37fcc5d8d89e service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:22.100709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872094e-cad0-4015-b644-37fcc5d8d89e service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:22.100994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872094e-cad0-4015-b644-37fcc5d8d89e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:22.250111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872094e-cad0-4015-b644-37fcc5d8d89e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:22.252204 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1d07f-8200-44f8-a5b3-5018d1ffe0a5 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:22.252722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872094e-cad0-4015-b644-37fcc5d8d89e service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:22.259444 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb0757-5802-4c8a-abc9-4b59dd1742fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:22.260420 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb0757-5802-4c8a-abc9-4b59dd1742fd service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:22.260865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb0757-5802-4c8a-abc9-4b59dd1742fd service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:22.261285 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb0757-5802-4c8a-abc9-4b59dd1742fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:22.284007 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 842/1697] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:22.285762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872094e-cad0-4015-b644-37fcc5d8d89e service nova] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:22.303463 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:22.312787 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872094e-cad0-4015-b644-37fcc5d8d89e service nova] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:22.337905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72267df6-e546-46ba-a68f-facb72e50a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:22.338625 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72267df6-e546-46ba-a68f-facb72e50a7c admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:22.338913 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72267df6-e546-46ba-a68f-facb72e50a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:22.339188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72267df6-e546-46ba-a68f-facb72e50a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:22.339926 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72267df6-e546-46ba-a68f-facb72e50a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:22.341612 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72267df6-e546-46ba-a68f-facb72e50a7c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:22.402409 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb0757-5802-4c8a-abc9-4b59dd1742fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:22.402409 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb0757-5802-4c8a-abc9-4b59dd1742fd service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:22.409493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 843/1698] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:22.409714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 856/1699] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:22.412528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72267df6-e546-46ba-a68f-facb72e50a7c admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:22.437326 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72267df6-e546-46ba-a68f-facb72e50a7c admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:22.443488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931db1ad-c11e-4a71-8807-55ba988bdab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:22.444588 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931db1ad-c11e-4a71-8807-55ba988bdab4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:22.444876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931db1ad-c11e-4a71-8807-55ba988bdab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:22.445410 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931db1ad-c11e-4a71-8807-55ba988bdab4 admin admin] PATH_INFO: `/v3/projects/3b4d7c88bdb94f9598daf42f9e37924a/users/1d9da597fcf6423dbe66af945445545a/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:22.466677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931db1ad-c11e-4a71-8807-55ba988bdab4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b4d7c88bdb94f9598daf42f9e37924a, user_id=1d9da597fcf6423dbe66af945445545a, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:22.470053 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931db1ad-c11e-4a71-8807-55ba988bdab4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:22.614107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 844/1700] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:22 2022] PUT /identity/v3/projects/3b4d7c88bdb94f9598daf42f9e37924a/users/1d9da597fcf6423dbe66af945445545a/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:22.616749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb0757-5802-4c8a-abc9-4b59dd1742fd service nova] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:22.656668 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb0757-5802-4c8a-abc9-4b59dd1742fd service nova] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:22.842847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403059e-5c85-4df2-bae3-433d904e788c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:22.843540 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403059e-5c85-4df2-bae3-433d904e788c admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:22.844027 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403059e-5c85-4df2-bae3-433d904e788c admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:22.844315 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403059e-5c85-4df2-bae3-433d904e788c admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:22.845264 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403059e-5c85-4df2-bae3-433d904e788c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:22.847035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403059e-5c85-4df2-bae3-433d904e788c admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:22.860157 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 857/1701] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:22.864523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931db1ad-c11e-4a71-8807-55ba988bdab4 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:22.888086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931db1ad-c11e-4a71-8807-55ba988bdab4 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:22.902366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b31d92-a613-4ba8-9ce1-083d85a03748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:22.905514 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b31d92-a613-4ba8-9ce1-083d85a03748 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:22.905932 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b31d92-a613-4ba8-9ce1-083d85a03748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:22.906322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b31d92-a613-4ba8-9ce1-083d85a03748 admin admin] PATH_INFO: `/v3/projects/3b4d7c88bdb94f9598daf42f9e37924a/users/1d9da597fcf6423dbe66af945445545a/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:22.915273 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b31d92-a613-4ba8-9ce1-083d85a03748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b4d7c88bdb94f9598daf42f9e37924a, user_id=1d9da597fcf6423dbe66af945445545a, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:22.923938 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b31d92-a613-4ba8-9ce1-083d85a03748 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:22.963614 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 845/1702] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:22 2022] PUT /identity/v3/projects/3b4d7c88bdb94f9598daf42f9e37924a/users/1d9da597fcf6423dbe66af945445545a/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:22.977970 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c142e9f-c408-4551-b89d-21c5f0460cfc None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:22.979172 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c142e9f-c408-4551-b89d-21c5f0460cfc None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:22.979578 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c142e9f-c408-4551-b89d-21c5f0460cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:23.050024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-2c142e9f-c408-4551-b89d-21c5f0460cfc None None] MFA Rules not processed for user `1d9da597fcf6423dbe66af945445545a`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:23.051763 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-2c142e9f-c408-4551-b89d-21c5f0460cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:23.072136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:23.243059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b31d92-a613-4ba8-9ce1-083d85a03748 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:23.245297 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-61b31d92-a613-4ba8-9ce1-083d85a03748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:23.269564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 858/1703] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:23.280374 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c142e9f-c408-4551-b89d-21c5f0460cfc None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:23.282896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-2c142e9f-c408-4551-b89d-21c5f0460cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:23.364539 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b31d92-a613-4ba8-9ce1-083d85a03748 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:23.371457 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8b799-b5c2-4e7f-a0f0-8186ecf5e34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:23.372812 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8b799-b5c2-4e7f-a0f0-8186ecf5e34f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:23.374249 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8b799-b5c2-4e7f-a0f0-8186ecf5e34f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:23.374871 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8b799-b5c2-4e7f-a0f0-8186ecf5e34f admin admin] PATH_INFO: `/v3/users/72cb75b4931b42f6bd5f71fa0a99a15e` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:23.376562 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8b799-b5c2-4e7f-a0f0-8186ecf5e34f admin admin] RBAC: Authorizing `identity:delete_user(user_id=72cb75b4931b42f6bd5f71fa0a99a15e)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:23.378835 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8b799-b5c2-4e7f-a0f0-8186ecf5e34f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:23.456595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c142e9f-c408-4551-b89d-21c5f0460cfc None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:23.471560 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45179b1-1167-4631-b129-e803e776ec8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:23.474947 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45179b1-1167-4631-b129-e803e776ec8d service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:23.475521 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45179b1-1167-4631-b129-e803e776ec8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:23.482170 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45179b1-1167-4631-b129-e803e776ec8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:23.524212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45179b1-1167-4631-b129-e803e776ec8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:23.529287 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45179b1-1167-4631-b129-e803e776ec8d service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:23.576267 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 859/1704] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:23.649937 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1fb8b799-b5c2-4e7f-a0f0-8186ecf5e34f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72cb75b4931b42f6bd5f71fa0a99a15e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:23.668934 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1fb8b799-b5c2-4e7f-a0f0-8186ecf5e34f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72cb75b4931b42f6bd5f71fa0a99a15e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:23.670904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1fb8b799-b5c2-4e7f-a0f0-8186ecf5e34f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72cb75b4931b42f6bd5f71fa0a99a15e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:23.675757 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1fb8b799-b5c2-4e7f-a0f0-8186ecf5e34f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72cb75b4931b42f6bd5f71fa0a99a15e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:23.709206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1fb8b799-b5c2-4e7f-a0f0-8186ecf5e34f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72cb75b4931b42f6bd5f71fa0a99a15e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:23.843014 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 846/1705] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:23 2022] DELETE /identity/v3/users/72cb75b4931b42f6bd5f71fa0a99a15e => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:23.844937 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd70943-4287-477b-9cbe-fdd9865c3088 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:23.845352 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd70943-4287-477b-9cbe-fdd9865c3088 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:23.845688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd70943-4287-477b-9cbe-fdd9865c3088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:23.857301 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8b799-b5c2-4e7f-a0f0-8186ecf5e34f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:23.872428 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1fb8b799-b5c2-4e7f-a0f0-8186ecf5e34f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:23.917735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-cfd70943-4287-477b-9cbe-fdd9865c3088 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:23.919483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-cfd70943-4287-477b-9cbe-fdd9865c3088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:23.973107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:24.022800 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8b799-b5c2-4e7f-a0f0-8186ecf5e34f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:24.042378 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7f1b8-00fe-4c14-8762-98966bdbe4a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:24.044431 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7f1b8-00fe-4c14-8762-98966bdbe4a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:24.049366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7f1b8-00fe-4c14-8762-98966bdbe4a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:24.049366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7f1b8-00fe-4c14-8762-98966bdbe4a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:24.052808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 860/1706] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:24.058306 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-01d7f1b8-00fe-4c14-8762-98966bdbe4a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:24.081489 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd70943-4287-477b-9cbe-fdd9865c3088 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:24.085175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-cfd70943-4287-477b-9cbe-fdd9865c3088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:24.222259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7f1b8-00fe-4c14-8762-98966bdbe4a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:24.223867 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7f1b8-00fe-4c14-8762-98966bdbe4a0 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:24.249491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 847/1707] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:24.260026 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd70943-4287-477b-9cbe-fdd9865c3088 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:24.282595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffdc39-b673-4f55-996d-85c1edb43f38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:24.286584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffdc39-b673-4f55-996d-85c1edb43f38 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:24.286584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffdc39-b673-4f55-996d-85c1edb43f38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:24.286584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffdc39-b673-4f55-996d-85c1edb43f38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:24.328722 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ffdc39-b673-4f55-996d-85c1edb43f38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:24.339214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ffdc39-b673-4f55-996d-85c1edb43f38 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:24.352841 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:24.369611 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 861/1708] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:25.151307 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8defd3-2aaf-4449-82e8-5ed68220dd5f None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:25.151891 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8defd3-2aaf-4449-82e8-5ed68220dd5f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:25.169196 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8defd3-2aaf-4449-82e8-5ed68220dd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:25.209347 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3e8defd3-2aaf-4449-82e8-5ed68220dd5f None None] MFA Rules not processed for user `030614ef56f143f8be6fcc0445ec9568`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:25.210705 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3e8defd3-2aaf-4449-82e8-5ed68220dd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:25.338861 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffdc39-b673-4f55-996d-85c1edb43f38 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:25.340670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-15ffdc39-b673-4f55-996d-85c1edb43f38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:25.373619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 848/1709] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 23 11:44:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:25.376755 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffdc39-b673-4f55-996d-85c1edb43f38 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:25.386906 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f732a-1536-4a5c-8b74-3fe7c84c430f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:25.388140 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f732a-1536-4a5c-8b74-3fe7c84c430f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:25.389307 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f732a-1536-4a5c-8b74-3fe7c84c430f admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:25.389887 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f732a-1536-4a5c-8b74-3fe7c84c430f admin admin] PATH_INFO: `/v3/projects/4431791e3ce94f4cae4d648f1af2f186` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:25.391732 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f732a-1536-4a5c-8b74-3fe7c84c430f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4431791e3ce94f4cae4d648f1af2f186)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:25.394153 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f732a-1536-4a5c-8b74-3fe7c84c430f admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:25.610867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-462f732a-1536-4a5c-8b74-3fe7c84c430f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4431791e3ce94f4cae4d648f1af2f186', 'request_id': 'req-462f732a-1536-4a5c-8b74-3fe7c84c430f'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:25.633419 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-462f732a-1536-4a5c-8b74-3fe7c84c430f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4431791e3ce94f4cae4d648f1af2f186', 'request_id': 'req-462f732a-1536-4a5c-8b74-3fe7c84c430f'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:25.799422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-462f732a-1536-4a5c-8b74-3fe7c84c430f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4431791e3ce94f4cae4d648f1af2f186', 'request_id': 'req-462f732a-1536-4a5c-8b74-3fe7c84c430f'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:25.821740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-462f732a-1536-4a5c-8b74-3fe7c84c430f admin admin] The token cache is being invalidate because project 4431791e3ce94f4cae4d648f1af2f186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:25.822238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-462f732a-1536-4a5c-8b74-3fe7c84c430f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:25.823605 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-462f732a-1536-4a5c-8b74-3fe7c84c430f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:25.832134 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 862/1710] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:25 2022] DELETE /identity/v3/projects/4431791e3ce94f4cae4d648f1af2f186 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:25.834385 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19df14-530e-461d-85f9-f628dc8b32e5 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:25.834509 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19df14-530e-461d-85f9-f628dc8b32e5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:25.834777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19df14-530e-461d-85f9-f628dc8b32e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:25.917320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-df19df14-530e-461d-85f9-f628dc8b32e5 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:25.918662 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-df19df14-530e-461d-85f9-f628dc8b32e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:25.934995 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f732a-1536-4a5c-8b74-3fe7c84c430f admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:25.937557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-462f732a-1536-4a5c-8b74-3fe7c84c430f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:26.111022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f732a-1536-4a5c-8b74-3fe7c84c430f admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:26.124273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f56767-19eb-4ff3-87d7-0b7c2522d772 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450b1fad96604fb09d0705895c6d3da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:26.143731 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f56767-19eb-4ff3-87d7-0b7c2522d772 service swift] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:26.145995 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f56767-19eb-4ff3-87d7-0b7c2522d772 service swift] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:26.147209 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f56767-19eb-4ff3-87d7-0b7c2522d772 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:26.152703 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-10f56767-19eb-4ff3-87d7-0b7c2522d772 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:26.165151 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 849/1711] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:26.167599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef05b9-229f-439d-82ce-2c5a495cb7ad None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:26.167599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef05b9-229f-439d-82ce-2c5a495cb7ad None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:26.167599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef05b9-229f-439d-82ce-2c5a495cb7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:26.252220 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-89ef05b9-229f-439d-82ce-2c5a495cb7ad None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:26.260095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-89ef05b9-229f-439d-82ce-2c5a495cb7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:26.296795 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 850/1712] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:44:26.310835 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef05b9-229f-439d-82ce-2c5a495cb7ad None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:26.330735 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef05b9-229f-439d-82ce-2c5a495cb7ad None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:26.354305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f46cc8-ae97-4c66-9c85-03ded359f9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:26.357980 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f46cc8-ae97-4c66-9c85-03ded359f9cc admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:26.364406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f46cc8-ae97-4c66-9c85-03ded359f9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:26.364894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f46cc8-ae97-4c66-9c85-03ded359f9cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:26.371743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f46cc8-ae97-4c66-9c85-03ded359f9cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:26.372674 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f46cc8-ae97-4c66-9c85-03ded359f9cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:26.375498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f46cc8-ae97-4c66-9c85-03ded359f9cc admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:26.383554 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f56767-19eb-4ff3-87d7-0b7c2522d772 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:26.387246 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f56767-19eb-4ff3-87d7-0b7c2522d772 service swift] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:26.403978 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 851/1713] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:44:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:44:26.408285 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f46cc8-ae97-4c66-9c85-03ded359f9cc admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:26.415596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 863/1714] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 23 11:44:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:44:26.415596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f56767-19eb-4ff3-87d7-0b7c2522d772 service swift] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:26.416984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-26f46cc8-ae97-4c66-9c85-03ded359f9cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:26.456507 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f56767-19eb-4ff3-87d7-0b7c2522d772 service swift] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:26.461497 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f46cc8-ae97-4c66-9c85-03ded359f9cc admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:26.469189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1329f-9ea0-486c-bb98-2f6a7fb9fa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:26.470350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1329f-9ea0-486c-bb98-2f6a7fb9fa95 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:26.471954 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1329f-9ea0-486c-bb98-2f6a7fb9fa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:26.472763 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bd2ef-5b87-4616-974d-72577f707239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:26.473165 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1329f-9ea0-486c-bb98-2f6a7fb9fa95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:26.474572 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bd2ef-5b87-4616-974d-72577f707239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:26.475097 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1329f-9ea0-486c-bb98-2f6a7fb9fa95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:26.477123 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1329f-9ea0-486c-bb98-2f6a7fb9fa95 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:26.477558 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bd2ef-5b87-4616-974d-72577f707239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:26.478029 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bd2ef-5b87-4616-974d-72577f707239 admin admin] PATH_INFO: `/v3/users/b4f455bc358f4faaa702c96d4818aeb2` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:26.480959 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00bd2ef-5b87-4616-974d-72577f707239 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4f455bc358f4faaa702c96d4818aeb2)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:26.485446 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00bd2ef-5b87-4616-974d-72577f707239 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:26.524278 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:26.633567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:26.653556 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 864/1715] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:26 2022] POST /identity/v3/projects => generated 355 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:26.683081 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1329f-9ea0-486c-bb98-2f6a7fb9fa95 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:26.747868 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e00bd2ef-5b87-4616-974d-72577f707239 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4f455bc358f4faaa702c96d4818aeb2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:26.798811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e00bd2ef-5b87-4616-974d-72577f707239 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4f455bc358f4faaa702c96d4818aeb2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:26.801122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e00bd2ef-5b87-4616-974d-72577f707239 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4f455bc358f4faaa702c96d4818aeb2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:26.803276 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e00bd2ef-5b87-4616-974d-72577f707239 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4f455bc358f4faaa702c96d4818aeb2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:26.868242 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e00bd2ef-5b87-4616-974d-72577f707239 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4f455bc358f4faaa702c96d4818aeb2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:26.957774 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1329f-9ea0-486c-bb98-2f6a7fb9fa95 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:26.965171 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c123ec-77c4-4642-b976-ffd4a3336465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:26.965896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c123ec-77c4-4642-b976-ffd4a3336465 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:26.966179 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c123ec-77c4-4642-b976-ffd4a3336465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:26.966457 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c123ec-77c4-4642-b976-ffd4a3336465 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:26.968165 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c123ec-77c4-4642-b976-ffd4a3336465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:26.977378 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c123ec-77c4-4642-b976-ffd4a3336465 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:27.049367 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 852/1716] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:26 2022] DELETE /identity/v3/users/b4f455bc358f4faaa702c96d4818aeb2 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:27.060229 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186fb5c-56bf-456b-bf26-32cc70480625 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:27.060682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186fb5c-56bf-456b-bf26-32cc70480625 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:27.061126 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186fb5c-56bf-456b-bf26-32cc70480625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:27.104148 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5186fb5c-56bf-456b-bf26-32cc70480625 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:27.105429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5186fb5c-56bf-456b-bf26-32cc70480625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:27.135599 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 865/1717] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:26 2022] POST /identity/v3/users => generated 278 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:27.137812 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c123ec-77c4-4642-b976-ffd4a3336465 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:27.140311 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f7c123ec-77c4-4642-b976-ffd4a3336465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:27.208163 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c123ec-77c4-4642-b976-ffd4a3336465 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:27.462283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 853/1718] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:27.502975 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186fb5c-56bf-456b-bf26-32cc70480625 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:27.504466 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5186fb5c-56bf-456b-bf26-32cc70480625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:27.605319 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bd436-e1e7-4067-930a-887b1481a0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:27.607148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bd436-e1e7-4067-930a-887b1481a0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:27.608864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bd436-e1e7-4067-930a-887b1481a0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:27.609382 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bd436-e1e7-4067-930a-887b1481a0f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:27.610843 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bd436-e1e7-4067-930a-887b1481a0f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:27.613170 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bd436-e1e7-4067-930a-887b1481a0f9 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:27.662624 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 866/1719] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:27 2022] GET /identity/v3/roles => generated 1420 bytes in 525 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:27.666715 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bd436-e1e7-4067-930a-887b1481a0f9 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:27.698044 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bd436-e1e7-4067-930a-887b1481a0f9 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:27.705095 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8583be-097c-4e5b-8b8a-88f545742f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:27.706613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8583be-097c-4e5b-8b8a-88f545742f23 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:27.707012 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8583be-097c-4e5b-8b8a-88f545742f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:27.707387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8583be-097c-4e5b-8b8a-88f545742f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:27.708239 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8583be-097c-4e5b-8b8a-88f545742f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:27.710257 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8583be-097c-4e5b-8b8a-88f545742f23 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:27.727640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 867/1720] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:27 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:27.734899 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8583be-097c-4e5b-8b8a-88f545742f23 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:27.751050 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8583be-097c-4e5b-8b8a-88f545742f23 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:27.771605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186fb5c-56bf-456b-bf26-32cc70480625 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:27.777252 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e06eb-59f1-49fe-a75d-daaa1c5c2114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:27.788891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e06eb-59f1-49fe-a75d-daaa1c5c2114 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:27.789409 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e06eb-59f1-49fe-a75d-daaa1c5c2114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:27.789900 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e06eb-59f1-49fe-a75d-daaa1c5c2114 admin admin] PATH_INFO: `/v3/projects/68b3200de93347789acada3e464271d7/users/12840922e0a0477a92ff9be10d89e7c5/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:27.807869 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f183f2-3825-43da-97ac-65b4c9a126e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:27.816402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f183f2-3825-43da-97ac-65b4c9a126e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:27.816589 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e06eb-59f1-49fe-a75d-daaa1c5c2114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68b3200de93347789acada3e464271d7, user_id=12840922e0a0477a92ff9be10d89e7c5, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:27.817050 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f183f2-3825-43da-97ac-65b4c9a126e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:27.818116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f183f2-3825-43da-97ac-65b4c9a126e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:27.821252 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e06eb-59f1-49fe-a75d-daaa1c5c2114 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:27.863260 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f183f2-3825-43da-97ac-65b4c9a126e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:27.864786 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f183f2-3825-43da-97ac-65b4c9a126e6 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:27.922566 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 868/1721] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:27 2022] PUT /identity/v3/projects/68b3200de93347789acada3e464271d7/users/12840922e0a0477a92ff9be10d89e7c5/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:27.922816 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 854/1722] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:27.926363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682f4c0-e878-4e81-a280-37b75df98323 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:27.926731 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682f4c0-e878-4e81-a280-37b75df98323 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:27.927253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682f4c0-e878-4e81-a280-37b75df98323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:27.994143 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-9682f4c0-e878-4e81-a280-37b75df98323 None None] MFA Rules not processed for user `12840922e0a0477a92ff9be10d89e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:27.996437 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-9682f4c0-e878-4e81-a280-37b75df98323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:28.165483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 869/1723] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:27 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:28.170417 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f183f2-3825-43da-97ac-65b4c9a126e6 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:28.188653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f183f2-3825-43da-97ac-65b4c9a126e6 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:28.323549 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:28.461777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfaaa5-760b-4d24-91c3-b6a6f963b640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:28.462612 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfaaa5-760b-4d24-91c3-b6a6f963b640 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:28.463316 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfaaa5-760b-4d24-91c3-b6a6f963b640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:28.463316 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfaaa5-760b-4d24-91c3-b6a6f963b640 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:28.470000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfaaa5-760b-4d24-91c3-b6a6f963b640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:28.486031 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfaaa5-760b-4d24-91c3-b6a6f963b640 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:29.051576 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:29.072309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 855/1724] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:28 2022] POST /identity/v3/projects => generated 355 bytes in 903 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:29.077707 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682f4c0-e878-4e81-a280-37b75df98323 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:29.115942 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682f4c0-e878-4e81-a280-37b75df98323 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:29.361240 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f43311-1a8d-4d15-8105-3dc41e51fae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:29.361941 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f43311-1a8d-4d15-8105-3dc41e51fae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:29.362784 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f43311-1a8d-4d15-8105-3dc41e51fae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:29.365584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f43311-1a8d-4d15-8105-3dc41e51fae9 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:29.365584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f43311-1a8d-4d15-8105-3dc41e51fae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:29.372678 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f43311-1a8d-4d15-8105-3dc41e51fae9 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:29.578904 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 870/1725] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:29 2022] POST /identity/v3/users => generated 284 bytes in 503 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:29.585480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfaaa5-760b-4d24-91c3-b6a6f963b640 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:29.602358 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfaaa5-760b-4d24-91c3-b6a6f963b640 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:29.626481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2248d12-7be1-42c9-9802-27f36dcd9978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:29.626481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2248d12-7be1-42c9-9802-27f36dcd9978 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:29.626481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2248d12-7be1-42c9-9802-27f36dcd9978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:29.626481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2248d12-7be1-42c9-9802-27f36dcd9978 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:29.626795 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2248d12-7be1-42c9-9802-27f36dcd9978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:29.628849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2248d12-7be1-42c9-9802-27f36dcd9978 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:29.673829 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 856/1726] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:29.675271 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f43311-1a8d-4d15-8105-3dc41e51fae9 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:29.696970 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f43311-1a8d-4d15-8105-3dc41e51fae9 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:29.712933 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc50ee5-ec3b-45c5-b26e-2b33c9626f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:29.715037 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc50ee5-ec3b-45c5-b26e-2b33c9626f22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:29.715801 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc50ee5-ec3b-45c5-b26e-2b33c9626f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:29.742365 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc50ee5-ec3b-45c5-b26e-2b33c9626f22 admin admin] PATH_INFO: `/v3/projects/974513e8db8a46fc848c6a364adb7e73/users/19a07e583d404b658435a58898780279/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:29.836620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc50ee5-ec3b-45c5-b26e-2b33c9626f22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=974513e8db8a46fc848c6a364adb7e73, user_id=19a07e583d404b658435a58898780279, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:29.850826 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc50ee5-ec3b-45c5-b26e-2b33c9626f22 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:29.985878 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488ec39-4e38-4cf1-a482-9e64bd92e98a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:29.988243 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488ec39-4e38-4cf1-a482-9e64bd92e98a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:29.988243 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488ec39-4e38-4cf1-a482-9e64bd92e98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:29.998846 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 871/1727] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:29 2022] PUT /identity/v3/projects/974513e8db8a46fc848c6a364adb7e73/users/19a07e583d404b658435a58898780279/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:30.003564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc50ee5-ec3b-45c5-b26e-2b33c9626f22 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:30.042580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-1488ec39-4e38-4cf1-a482-9e64bd92e98a None None] MFA Rules not processed for user `810fa3f54cd04f989252765f7e76e602`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:30.043825 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1488ec39-4e38-4cf1-a482-9e64bd92e98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:30.221429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc50ee5-ec3b-45c5-b26e-2b33c9626f22 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:30.227653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57414e-d8ce-4695-a3b5-ddb604ff124f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:30.228597 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57414e-d8ce-4695-a3b5-ddb604ff124f admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:30.229031 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57414e-d8ce-4695-a3b5-ddb604ff124f admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:30.229472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57414e-d8ce-4695-a3b5-ddb604ff124f admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:30.231430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b57414e-d8ce-4695-a3b5-ddb604ff124f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:30.233514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b57414e-d8ce-4695-a3b5-ddb604ff124f admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:30.247725 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 872/1728] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:30.253165 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57414e-d8ce-4695-a3b5-ddb604ff124f admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:30.256589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 857/1729] 10.222.0.46 () {58 vars in 1015 bytes} [Thu Jun 23 11:44:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:44:30.311010 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57414e-d8ce-4695-a3b5-ddb604ff124f admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:30.344481 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f0bf5-4c26-4bf7-9460-d760e86a69f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:30.346487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f0bf5-4c26-4bf7-9460-d760e86a69f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:30.348519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f0bf5-4c26-4bf7-9460-d760e86a69f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:30.349301 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f0bf5-4c26-4bf7-9460-d760e86a69f1 admin admin] PATH_INFO: `/v3/projects/974513e8db8a46fc848c6a364adb7e73/users/19a07e583d404b658435a58898780279/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:30.357370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f0bf5-4c26-4bf7-9460-d760e86a69f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=974513e8db8a46fc848c6a364adb7e73, user_id=19a07e583d404b658435a58898780279, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:30.367299 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f0bf5-4c26-4bf7-9460-d760e86a69f1 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:30.409004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 873/1730] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:30 2022] PUT /identity/v3/projects/974513e8db8a46fc848c6a364adb7e73/users/19a07e583d404b658435a58898780279/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:30.409695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65270d4b-410a-4fd3-88df-14ab617d9b70 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:30.410101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65270d4b-410a-4fd3-88df-14ab617d9b70 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:30.413684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65270d4b-410a-4fd3-88df-14ab617d9b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:30.525003 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-65270d4b-410a-4fd3-88df-14ab617d9b70 None None] MFA Rules not processed for user `19a07e583d404b658435a58898780279`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:30.531302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-65270d4b-410a-4fd3-88df-14ab617d9b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:30.562357 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f0bf5-4c26-4bf7-9460-d760e86a69f1 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:30.565007 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-779f0bf5-4c26-4bf7-9460-d760e86a69f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:30.680109 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:30.780869 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 858/1731] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:30 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:30.781838 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65270d4b-410a-4fd3-88df-14ab617d9b70 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:30.833456 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f0bf5-4c26-4bf7-9460-d760e86a69f1 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:30.868893 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d60e23-1ba8-4e12-b347-25ba6fe19277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:30.876276 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:30.876426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d60e23-1ba8-4e12-b347-25ba6fe19277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:30.876426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d60e23-1ba8-4e12-b347-25ba6fe19277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:30.876426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d60e23-1ba8-4e12-b347-25ba6fe19277 admin admin] PATH_INFO: `/v3/users/c5b332d141e54132a169dcd755032925` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:30.885921 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d60e23-1ba8-4e12-b347-25ba6fe19277 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5b332d141e54132a169dcd755032925)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:30.894997 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d60e23-1ba8-4e12-b347-25ba6fe19277 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:30.933899 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65270d4b-410a-4fd3-88df-14ab617d9b70 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:30.949226 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10254c7d-6045-4597-b300-19e2e1907bab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:30.949922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10254c7d-6045-4597-b300-19e2e1907bab service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:30.950209 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10254c7d-6045-4597-b300-19e2e1907bab service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:30.950499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10254c7d-6045-4597-b300-19e2e1907bab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:30.986494 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10254c7d-6045-4597-b300-19e2e1907bab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:30.990480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10254c7d-6045-4597-b300-19e2e1907bab service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:31.018977 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 859/1732] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:31.021263 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10254c7d-6045-4597-b300-19e2e1907bab service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:31.026090 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-10254c7d-6045-4597-b300-19e2e1907bab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:31.150006 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10254c7d-6045-4597-b300-19e2e1907bab service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:31.153934 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf9e00-fce9-47f9-a929-782228451fee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:31.155524 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf9e00-fce9-47f9-a929-782228451fee service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:31.155800 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf9e00-fce9-47f9-a929-782228451fee service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:31.156230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf9e00-fce9-47f9-a929-782228451fee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:31.170355 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf9e00-fce9-47f9-a929-782228451fee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:31.173304 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf9e00-fce9-47f9-a929-782228451fee service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:31.192701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 860/1733] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:31 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:31.192701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf9e00-fce9-47f9-a929-782228451fee service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:31.194110 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-31bf9e00-fce9-47f9-a929-782228451fee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:31.229884 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf9e00-fce9-47f9-a929-782228451fee service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:31.233355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c3d60e23-1ba8-4e12-b347-25ba6fe19277 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5b332d141e54132a169dcd755032925'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:31.238353 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d908f-ab1c-4534-bc21-f43c3ec0c1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:31.240168 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d908f-ab1c-4534-bc21-f43c3ec0c1d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:31.240679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d908f-ab1c-4534-bc21-f43c3ec0c1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:31.241028 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d908f-ab1c-4534-bc21-f43c3ec0c1d3 admin admin] PATH_INFO: `/v3/projects/96a5cce3a55c4f819529ccb1cadb2b6a` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:31.244047 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d908f-ab1c-4534-bc21-f43c3ec0c1d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96a5cce3a55c4f819529ccb1cadb2b6a)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:31.246463 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d908f-ab1c-4534-bc21-f43c3ec0c1d3 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:31.262783 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c3d60e23-1ba8-4e12-b347-25ba6fe19277 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5b332d141e54132a169dcd755032925'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:31.271546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c3d60e23-1ba8-4e12-b347-25ba6fe19277 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5b332d141e54132a169dcd755032925'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:31.271546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c3d60e23-1ba8-4e12-b347-25ba6fe19277 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5b332d141e54132a169dcd755032925'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:31.317394 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c3d60e23-1ba8-4e12-b347-25ba6fe19277 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5b332d141e54132a169dcd755032925'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:31.460567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 874/1734] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:30 2022] DELETE /identity/v3/users/c5b332d141e54132a169dcd755032925 => generated 0 bytes in 896 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:31.468188 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae77d0-3bef-4efe-9fbd-ea0034933dd7 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:31.471175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae77d0-3bef-4efe-9fbd-ea0034933dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:31.471673 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae77d0-3bef-4efe-9fbd-ea0034933dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:31.514334 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cd3d908f-ab1c-4534-bc21-f43c3ec0c1d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96a5cce3a55c4f819529ccb1cadb2b6a', 'request_id': 'req-cd3d908f-ab1c-4534-bc21-f43c3ec0c1d3'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:31.567018 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cd3d908f-ab1c-4534-bc21-f43c3ec0c1d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96a5cce3a55c4f819529ccb1cadb2b6a', 'request_id': 'req-cd3d908f-ab1c-4534-bc21-f43c3ec0c1d3'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:31.641434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-cfae77d0-3bef-4efe-9fbd-ea0034933dd7 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:31.655718 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-cfae77d0-3bef-4efe-9fbd-ea0034933dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:31.781324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cd3d908f-ab1c-4534-bc21-f43c3ec0c1d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96a5cce3a55c4f819529ccb1cadb2b6a', 'request_id': 'req-cd3d908f-ab1c-4534-bc21-f43c3ec0c1d3'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:31.797695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cd3d908f-ab1c-4534-bc21-f43c3ec0c1d3 admin admin] The token cache is being invalidate because project 96a5cce3a55c4f819529ccb1cadb2b6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:31.798148 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cd3d908f-ab1c-4534-bc21-f43c3ec0c1d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:31.799059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cd3d908f-ab1c-4534-bc21-f43c3ec0c1d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:31.808436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 861/1735] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:31 2022] DELETE /identity/v3/projects/96a5cce3a55c4f819529ccb1cadb2b6a => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:31.830073 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d908f-ab1c-4534-bc21-f43c3ec0c1d3 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:31.839281 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-cd3d908f-ab1c-4534-bc21-f43c3ec0c1d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:31.912979 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 875/1736] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:31.914470 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae77d0-3bef-4efe-9fbd-ea0034933dd7 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:31.917246 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-cfae77d0-3bef-4efe-9fbd-ea0034933dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:31.977890 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae77d0-3bef-4efe-9fbd-ea0034933dd7 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:31.998850 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d908f-ab1c-4534-bc21-f43c3ec0c1d3 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:32.000537 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0953a13-830d-4707-9b89-7af61d828056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:32.005001 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0953a13-830d-4707-9b89-7af61d828056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:32.012639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0953a13-830d-4707-9b89-7af61d828056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:32.012639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0953a13-830d-4707-9b89-7af61d828056 admin admin] PATH_INFO: `/v3/users/92e9bede2f88450a881ba145c10bc739` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:32.016443 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0953a13-830d-4707-9b89-7af61d828056 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92e9bede2f88450a881ba145c10bc739)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:32.020805 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0953a13-830d-4707-9b89-7af61d828056 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:32.172158 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ec410-26c2-43a5-86d2-c62c91ee2527 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:32.175067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ec410-26c2-43a5-86d2-c62c91ee2527 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:32.176814 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ec410-26c2-43a5-86d2-c62c91ee2527 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:32.177856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ec410-26c2-43a5-86d2-c62c91ee2527 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:32.184831 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-fe2ec410-26c2-43a5-86d2-c62c91ee2527 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:32.308023 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:32.348856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:32.355486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f0953a13-830d-4707-9b89-7af61d828056 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92e9bede2f88450a881ba145c10bc739'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:32.358425 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ec410-26c2-43a5-86d2-c62c91ee2527 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:32.362803 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ec410-26c2-43a5-86d2-c62c91ee2527 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:32.387301 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f0953a13-830d-4707-9b89-7af61d828056 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92e9bede2f88450a881ba145c10bc739'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:32.390396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f0953a13-830d-4707-9b89-7af61d828056 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92e9bede2f88450a881ba145c10bc739'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:32.395709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f0953a13-830d-4707-9b89-7af61d828056 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92e9bede2f88450a881ba145c10bc739'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:32.396105 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 862/1737] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:31 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:32.417270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ec410-26c2-43a5-86d2-c62c91ee2527 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:32.423541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f0953a13-830d-4707-9b89-7af61d828056 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92e9bede2f88450a881ba145c10bc739'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:32.426273 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-fe2ec410-26c2-43a5-86d2-c62c91ee2527 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:32.550399 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ec410-26c2-43a5-86d2-c62c91ee2527 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:32.555476 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 876/1738] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:31 2022] DELETE /identity/v3/users/92e9bede2f88450a881ba145c10bc739 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:32.556946 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0953a13-830d-4707-9b89-7af61d828056 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:32.569799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f0953a13-830d-4707-9b89-7af61d828056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:32.723074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3ce6a-a6a4-4982-a987-c54485bc8bbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:32.728045 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3ce6a-a6a4-4982-a987-c54485bc8bbc service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:32.728045 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3ce6a-a6a4-4982-a987-c54485bc8bbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:32.728045 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3ce6a-a6a4-4982-a987-c54485bc8bbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:32.728045 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-81c3ce6a-a6a4-4982-a987-c54485bc8bbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:32.782146 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0953a13-830d-4707-9b89-7af61d828056 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:32.802154 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb106d3-72f0-4aca-a0ed-be8589c4b037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:32.803385 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb106d3-72f0-4aca-a0ed-be8589c4b037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:32.803698 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb106d3-72f0-4aca-a0ed-be8589c4b037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:32.803992 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb106d3-72f0-4aca-a0ed-be8589c4b037 admin admin] PATH_INFO: `/v3/users/4f5d9af6661c475b91f0da76e938c567` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:32.826643 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb106d3-72f0-4aca-a0ed-be8589c4b037 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f5d9af6661c475b91f0da76e938c567)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:32.826643 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb106d3-72f0-4aca-a0ed-be8589c4b037 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:32.964901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c3ce6a-a6a4-4982-a987-c54485bc8bbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:32.972606 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c3ce6a-a6a4-4982-a987-c54485bc8bbc service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:33.017829 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 863/1739] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:33.019890 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ed918-4978-4724-be01-d61a84d00706 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:33.020343 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ed918-4978-4724-be01-d61a84d00706 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:33.028351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ed918-4978-4724-be01-d61a84d00706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:33.077620 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-714ed918-4978-4724-be01-d61a84d00706 None None] MFA Rules not processed for user `d1b988f3241a46308b7c3683e2943b78`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:33.082961 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-714ed918-4978-4724-be01-d61a84d00706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:33.204606 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-deb106d3-72f0-4aca-a0ed-be8589c4b037 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f5d9af6661c475b91f0da76e938c567'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:33.239204 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-deb106d3-72f0-4aca-a0ed-be8589c4b037 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f5d9af6661c475b91f0da76e938c567'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:33.243218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-deb106d3-72f0-4aca-a0ed-be8589c4b037 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f5d9af6661c475b91f0da76e938c567'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:33.245273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-deb106d3-72f0-4aca-a0ed-be8589c4b037 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f5d9af6661c475b91f0da76e938c567'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:33.284386 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-deb106d3-72f0-4aca-a0ed-be8589c4b037 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f5d9af6661c475b91f0da76e938c567'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:33.331528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 864/1740] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:33.395708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 877/1741] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:32 2022] DELETE /identity/v3/users/4f5d9af6661c475b91f0da76e938c567 => generated 0 bytes in 839 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:33.408536 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7517bf63-672f-4357-b84a-e94a50032436 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:33.408536 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7517bf63-672f-4357-b84a-e94a50032436 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:33.408536 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7517bf63-672f-4357-b84a-e94a50032436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:33.457043 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-7517bf63-672f-4357-b84a-e94a50032436 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:33.458283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7517bf63-672f-4357-b84a-e94a50032436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:33.632722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 865/1742] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:33.769689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb106d3-72f0-4aca-a0ed-be8589c4b037 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:33.772801 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-deb106d3-72f0-4aca-a0ed-be8589c4b037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:33.884577 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb106d3-72f0-4aca-a0ed-be8589c4b037 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:33.899335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b3d4c-a284-43f0-af3f-ba9fd278d182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:33.900070 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b3d4c-a284-43f0-af3f-ba9fd278d182 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:33.900356 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b3d4c-a284-43f0-af3f-ba9fd278d182 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:33.900633 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b3d4c-a284-43f0-af3f-ba9fd278d182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:33.930729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b3d4c-a284-43f0-af3f-ba9fd278d182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:33.932276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b3d4c-a284-43f0-af3f-ba9fd278d182 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:33.946079 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 878/1743] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:34.359156 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:34.462803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:35.692663 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7517bf63-672f-4357-b84a-e94a50032436 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:35.694337 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7517bf63-672f-4357-b84a-e94a50032436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:35.714069 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7517bf63-672f-4357-b84a-e94a50032436 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:35.722269 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5ba72-9339-45ff-9ba5-3d5d039f2296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:35.722269 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5ba72-9339-45ff-9ba5-3d5d039f2296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:35.722269 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5ba72-9339-45ff-9ba5-3d5d039f2296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:35.722269 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5ba72-9339-45ff-9ba5-3d5d039f2296 admin admin] PATH_INFO: `/v3/projects/8ebf158378db42608ae39baaf29589f9` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:35.727011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d5ba72-9339-45ff-9ba5-3d5d039f2296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ebf158378db42608ae39baaf29589f9)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:35.735019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d5ba72-9339-45ff-9ba5-3d5d039f2296 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:35.814475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b3d4c-a284-43f0-af3f-ba9fd278d182 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:35.821129 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-be9b3d4c-a284-43f0-af3f-ba9fd278d182 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:35.860215 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b3d4c-a284-43f0-af3f-ba9fd278d182 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:35.928572 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73442f-d7c4-4336-ba3c-60e35697cd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:35.929705 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73442f-d7c4-4336-ba3c-60e35697cd98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:35.930671 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73442f-d7c4-4336-ba3c-60e35697cd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:35.932279 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73442f-d7c4-4336-ba3c-60e35697cd98 admin admin] PATH_INFO: `/v3/projects/df2920638b244cd2be4b797df67f8291` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:35.937270 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73442f-d7c4-4336-ba3c-60e35697cd98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df2920638b244cd2be4b797df67f8291)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:35.940214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73442f-d7c4-4336-ba3c-60e35697cd98 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:35.963595 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f7d5ba72-9339-45ff-9ba5-3d5d039f2296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ebf158378db42608ae39baaf29589f9', 'request_id': 'req-f7d5ba72-9339-45ff-9ba5-3d5d039f2296'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:35.968639 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f7d5ba72-9339-45ff-9ba5-3d5d039f2296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ebf158378db42608ae39baaf29589f9', 'request_id': 'req-f7d5ba72-9339-45ff-9ba5-3d5d039f2296'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.073971 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f7d5ba72-9339-45ff-9ba5-3d5d039f2296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ebf158378db42608ae39baaf29589f9', 'request_id': 'req-f7d5ba72-9339-45ff-9ba5-3d5d039f2296'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.101106 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f7d5ba72-9339-45ff-9ba5-3d5d039f2296 admin admin] The token cache is being invalidate because project 8ebf158378db42608ae39baaf29589f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:36.101396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f7d5ba72-9339-45ff-9ba5-3d5d039f2296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.109091 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f7d5ba72-9339-45ff-9ba5-3d5d039f2296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.114807 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 866/1744] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:35 2022] DELETE /identity/v3/projects/8ebf158378db42608ae39baaf29589f9 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:36.120129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba76b37-234c-4192-af71-ab8114d58182 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.120475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba76b37-234c-4192-af71-ab8114d58182 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.121009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba76b37-234c-4192-af71-ab8114d58182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.179687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-6ba76b37-234c-4192-af71-ab8114d58182 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:36.184661 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6ba76b37-234c-4192-af71-ab8114d58182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:36.201793 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7c73442f-d7c4-4336-ba3c-60e35697cd98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df2920638b244cd2be4b797df67f8291', 'request_id': 'req-7c73442f-d7c4-4336-ba3c-60e35697cd98'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.234721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7c73442f-d7c4-4336-ba3c-60e35697cd98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df2920638b244cd2be4b797df67f8291', 'request_id': 'req-7c73442f-d7c4-4336-ba3c-60e35697cd98'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.272424 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:36.330008 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7c73442f-d7c4-4336-ba3c-60e35697cd98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df2920638b244cd2be4b797df67f8291', 'request_id': 'req-7c73442f-d7c4-4336-ba3c-60e35697cd98'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.352773 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7c73442f-d7c4-4336-ba3c-60e35697cd98 admin admin] The token cache is being invalidate because project df2920638b244cd2be4b797df67f8291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:36.354256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7c73442f-d7c4-4336-ba3c-60e35697cd98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.358566 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7c73442f-d7c4-4336-ba3c-60e35697cd98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.363866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 879/1745] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:35 2022] DELETE /identity/v3/projects/df2920638b244cd2be4b797df67f8291 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:36.368293 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d771e3-f712-4bfc-9c6f-4a8155d4b6b7 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.368924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d771e3-f712-4bfc-9c6f-4a8155d4b6b7 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.370053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d771e3-f712-4bfc-9c6f-4a8155d4b6b7 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.377950 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 880/1746] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 23 11:44:36 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:44:36.377950 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d771e3-f712-4bfc-9c6f-4a8155d4b6b7 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:36.386363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e3d771e3-f712-4bfc-9c6f-4a8155d4b6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:36.446139 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 867/1747] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:36.452221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b963706-3b16-4350-a225-e82c036650c0 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.452491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b963706-3b16-4350-a225-e82c036650c0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.452780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b963706-3b16-4350-a225-e82c036650c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.512240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-7b963706-3b16-4350-a225-e82c036650c0 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:36.512240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7b963706-3b16-4350-a225-e82c036650c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:36.522787 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d771e3-f712-4bfc-9c6f-4a8155d4b6b7 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:36.529494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f332455-241a-4d52-9ddb-d8af7bd44799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:36.531197 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f332455-241a-4d52-9ddb-d8af7bd44799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.531692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f332455-241a-4d52-9ddb-d8af7bd44799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.532185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f332455-241a-4d52-9ddb-d8af7bd44799 admin admin] PATH_INFO: `/v3/users/1d9da597fcf6423dbe66af945445545a` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.534012 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f332455-241a-4d52-9ddb-d8af7bd44799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d9da597fcf6423dbe66af945445545a)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:36.536330 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f332455-241a-4d52-9ddb-d8af7bd44799 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:36.566810 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 868/1748] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:36.575135 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af2e86-a7f2-43be-ac08-97163a2f21c5 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.576780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af2e86-a7f2-43be-ac08-97163a2f21c5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.576915 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af2e86-a7f2-43be-ac08-97163a2f21c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.624129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b7af2e86-a7f2-43be-ac08-97163a2f21c5 None None] MFA Rules not processed for user `161c5ed1e47e450290a1f428ee100b68`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:36.625954 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b7af2e86-a7f2-43be-ac08-97163a2f21c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:36.817367 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 869/1749] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 23 11:44:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:36.818981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af2e86-a7f2-43be-ac08-97163a2f21c5 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:36.832811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af2e86-a7f2-43be-ac08-97163a2f21c5 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:36.857902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd589c8-61f5-4763-9968-775cf54696a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:36.865515 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3f332455-241a-4d52-9ddb-d8af7bd44799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d9da597fcf6423dbe66af945445545a'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.868013 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd589c8-61f5-4763-9968-775cf54696a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.868450 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd589c8-61f5-4763-9968-775cf54696a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.868626 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd589c8-61f5-4763-9968-775cf54696a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.869690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd589c8-61f5-4763-9968-775cf54696a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:36.869904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd589c8-61f5-4763-9968-775cf54696a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:36.879212 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd589c8-61f5-4763-9968-775cf54696a7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:36.886149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3f332455-241a-4d52-9ddb-d8af7bd44799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d9da597fcf6423dbe66af945445545a'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.887230 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3f332455-241a-4d52-9ddb-d8af7bd44799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d9da597fcf6423dbe66af945445545a'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.888361 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3f332455-241a-4d52-9ddb-d8af7bd44799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d9da597fcf6423dbe66af945445545a'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.906423 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:36.911782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3f332455-241a-4d52-9ddb-d8af7bd44799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d9da597fcf6423dbe66af945445545a'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.948805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 870/1750] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:44:36.950908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd589c8-61f5-4763-9968-775cf54696a7 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:36.956465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9cd589c8-61f5-4763-9968-775cf54696a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:36.985620 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd589c8-61f5-4763-9968-775cf54696a7 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:36.995557 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 881/1751] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:36 2022] DELETE /identity/v3/users/1d9da597fcf6423dbe66af945445545a => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:36.999380 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d8df5-1a5e-492d-8d57-f3fbfc9f6714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:37.000270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d8df5-1a5e-492d-8d57-f3fbfc9f6714 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:37.001516 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d8df5-1a5e-492d-8d57-f3fbfc9f6714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:37.002035 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d8df5-1a5e-492d-8d57-f3fbfc9f6714 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:37.004606 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d8df5-1a5e-492d-8d57-f3fbfc9f6714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:37.008514 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d8df5-1a5e-492d-8d57-f3fbfc9f6714 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:37.065704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f332455-241a-4d52-9ddb-d8af7bd44799 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:37.067639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-3f332455-241a-4d52-9ddb-d8af7bd44799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:37.143883 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 871/1752] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:36 2022] POST /identity/v3/projects => generated 397 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:37.145775 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d8df5-1a5e-492d-8d57-f3fbfc9f6714 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:37.199899 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f332455-241a-4d52-9ddb-d8af7bd44799 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:37.348481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d8df5-1a5e-492d-8d57-f3fbfc9f6714 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:37.368506 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066edb0-9761-4f57-bc94-3f9dec81fff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:37.369198 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066edb0-9761-4f57-bc94-3f9dec81fff1 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:37.369512 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066edb0-9761-4f57-bc94-3f9dec81fff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:37.369742 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066edb0-9761-4f57-bc94-3f9dec81fff1 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:37.370609 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066edb0-9761-4f57-bc94-3f9dec81fff1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:37.372569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066edb0-9761-4f57-bc94-3f9dec81fff1 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:37.454903 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1df125-2006-4f49-8f0c-a262f4642808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:37.459357 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1df125-2006-4f49-8f0c-a262f4642808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:37.460309 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1df125-2006-4f49-8f0c-a262f4642808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:37.461385 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1df125-2006-4f49-8f0c-a262f4642808 admin admin] PATH_INFO: `/v3/projects/cda6be175c8f45b196c36e6b89aa6d1e` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:37.465058 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1df125-2006-4f49-8f0c-a262f4642808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cda6be175c8f45b196c36e6b89aa6d1e)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:37.475445 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1df125-2006-4f49-8f0c-a262f4642808 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:37.542089 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 872/1753] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:37 2022] POST /identity/v3/users => generated 299 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:37.543262 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066edb0-9761-4f57-bc94-3f9dec81fff1 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:37.556196 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9066edb0-9761-4f57-bc94-3f9dec81fff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:37.699230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066edb0-9761-4f57-bc94-3f9dec81fff1 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:37.705558 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df802e-289a-417e-8eb1-11538568aac9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:37.706521 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df802e-289a-417e-8eb1-11538568aac9 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:37.707070 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df802e-289a-417e-8eb1-11538568aac9 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:37.707532 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df802e-289a-417e-8eb1-11538568aac9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:37.710445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d8df802e-289a-417e-8eb1-11538568aac9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:37.727722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df802e-289a-417e-8eb1-11538568aac9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:37.730001 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df802e-289a-417e-8eb1-11538568aac9 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:37.744868 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 873/1754] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:37.746855 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df802e-289a-417e-8eb1-11538568aac9 service nova] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:37.755672 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-5c1df125-2006-4f49-8f0c-a262f4642808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cda6be175c8f45b196c36e6b89aa6d1e', 'request_id': 'req-5c1df125-2006-4f49-8f0c-a262f4642808'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:37.775291 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df802e-289a-417e-8eb1-11538568aac9 service nova] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:37.777621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-5c1df125-2006-4f49-8f0c-a262f4642808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cda6be175c8f45b196c36e6b89aa6d1e', 'request_id': 'req-5c1df125-2006-4f49-8f0c-a262f4642808'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:37.891452 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076a0c4-3582-4f75-88b3-17630a461c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:37.892217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076a0c4-3582-4f75-88b3-17630a461c45 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:37.892502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076a0c4-3582-4f75-88b3-17630a461c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:37.893260 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076a0c4-3582-4f75-88b3-17630a461c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:37.895881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076a0c4-3582-4f75-88b3-17630a461c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:37.897700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076a0c4-3582-4f75-88b3-17630a461c45 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:37.923238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 874/1755] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:37.927164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076a0c4-3582-4f75-88b3-17630a461c45 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:37.930481 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-5c1df125-2006-4f49-8f0c-a262f4642808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cda6be175c8f45b196c36e6b89aa6d1e', 'request_id': 'req-5c1df125-2006-4f49-8f0c-a262f4642808'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:37.946959 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076a0c4-3582-4f75-88b3-17630a461c45 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:37.949714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-5c1df125-2006-4f49-8f0c-a262f4642808 admin admin] The token cache is being invalidate because project cda6be175c8f45b196c36e6b89aa6d1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:37.950287 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-5c1df125-2006-4f49-8f0c-a262f4642808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:37.952944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-5c1df125-2006-4f49-8f0c-a262f4642808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:37.965839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 882/1756] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:37 2022] DELETE /identity/v3/projects/cda6be175c8f45b196c36e6b89aa6d1e => generated 0 bytes in 898 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:37.965987 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338c63f-0b76-4bbd-9b5a-5ee55907ab38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:37.967893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338c63f-0b76-4bbd-9b5a-5ee55907ab38 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:37.968175 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338c63f-0b76-4bbd-9b5a-5ee55907ab38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:37.968651 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338c63f-0b76-4bbd-9b5a-5ee55907ab38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:37.970201 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd6af6-3e63-4aa6-8384-fe48b4affa9a None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:37.970396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338c63f-0b76-4bbd-9b5a-5ee55907ab38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:37.972261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd6af6-3e63-4aa6-8384-fe48b4affa9a None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:37.972724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd6af6-3e63-4aa6-8384-fe48b4affa9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:37.973360 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338c63f-0b76-4bbd-9b5a-5ee55907ab38 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:37.994568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 875/1757] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:38.001554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338c63f-0b76-4bbd-9b5a-5ee55907ab38 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:38.004304 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6338c63f-0b76-4bbd-9b5a-5ee55907ab38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:38.015841 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-3ecd6af6-3e63-4aa6-8384-fe48b4affa9a None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:38.017538 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-3ecd6af6-3e63-4aa6-8384-fe48b4affa9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:38.021652 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338c63f-0b76-4bbd-9b5a-5ee55907ab38 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:38.027277 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be053dda-5a9e-4405-b265-6f8028953c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:38.028442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 883/1758] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:38.028608 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be053dda-5a9e-4405-b265-6f8028953c0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:38.029512 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be053dda-5a9e-4405-b265-6f8028953c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:38.030481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be053dda-5a9e-4405-b265-6f8028953c0d admin admin] PATH_INFO: `/v3/projects/220b20a472ef4bae838624c1994faf38/users/ae6348a5810c413d9bca62f4769a51e6/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:38.037072 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698a2e7-dead-4d2b-b9c0-eea8068e3536 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:38.037603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698a2e7-dead-4d2b-b9c0-eea8068e3536 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:38.038072 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698a2e7-dead-4d2b-b9c0-eea8068e3536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:38.055674 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be053dda-5a9e-4405-b265-6f8028953c0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=220b20a472ef4bae838624c1994faf38, user_id=ae6348a5810c413d9bca62f4769a51e6, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:38.058743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be053dda-5a9e-4405-b265-6f8028953c0d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:38.066823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-e698a2e7-dead-4d2b-b9c0-eea8068e3536 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:38.068463 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e698a2e7-dead-4d2b-b9c0-eea8068e3536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:38.079537 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 884/1759] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:38.084875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698a2e7-dead-4d2b-b9c0-eea8068e3536 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:38.097891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698a2e7-dead-4d2b-b9c0-eea8068e3536 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:38.104936 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f1523-978e-41cf-8c0f-5fdc91c788bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:38.105989 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f1523-978e-41cf-8c0f-5fdc91c788bd admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:38.106601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f1523-978e-41cf-8c0f-5fdc91c788bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:38.107348 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f1523-978e-41cf-8c0f-5fdc91c788bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:38.108630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f1523-978e-41cf-8c0f-5fdc91c788bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:38.109230 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f1523-978e-41cf-8c0f-5fdc91c788bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:38.111672 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f1523-978e-41cf-8c0f-5fdc91c788bd admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:38.136620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 885/1760] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:44:38.141080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f1523-978e-41cf-8c0f-5fdc91c788bd admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:38.160379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f1523-978e-41cf-8c0f-5fdc91c788bd admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:38.170872 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311a20c-b9ed-4df5-8164-e1b3f720548a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:38.172156 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311a20c-b9ed-4df5-8164-e1b3f720548a admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:38.172304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311a20c-b9ed-4df5-8164-e1b3f720548a admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:38.172619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311a20c-b9ed-4df5-8164-e1b3f720548a admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:38.173596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311a20c-b9ed-4df5-8164-e1b3f720548a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:38.179107 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311a20c-b9ed-4df5-8164-e1b3f720548a admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:38.204187 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 876/1761] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:38 2022] PUT /identity/v3/projects/220b20a472ef4bae838624c1994faf38/users/ae6348a5810c413d9bca62f4769a51e6/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:38.206074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a8d30-283f-40e1-b0ed-f9d706ef705c None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:38.206592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a8d30-283f-40e1-b0ed-f9d706ef705c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:38.207264 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a8d30-283f-40e1-b0ed-f9d706ef705c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:38.256294 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b45a8d30-283f-40e1-b0ed-f9d706ef705c None None] MFA Rules not processed for user `ae6348a5810c413d9bca62f4769a51e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:38.258996 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b45a8d30-283f-40e1-b0ed-f9d706ef705c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:38.373961 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 886/1762] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:38 2022] POST /identity/v3/projects => generated 375 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:38.377876 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311a20c-b9ed-4df5-8164-e1b3f720548a admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:38.383802 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 877/1763] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:38.388616 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a8d30-283f-40e1-b0ed-f9d706ef705c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:38.569267 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311a20c-b9ed-4df5-8164-e1b3f720548a admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:38.603900 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b21080-8ec4-43f2-bb65-5a967629e238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:38.610421 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a8d30-283f-40e1-b0ed-f9d706ef705c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:38.616243 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b21080-8ec4-43f2-bb65-5a967629e238 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:38.617614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b21080-8ec4-43f2-bb65-5a967629e238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:38.619766 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b21080-8ec4-43f2-bb65-5a967629e238 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:38.621244 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b21080-8ec4-43f2-bb65-5a967629e238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:38.624142 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b21080-8ec4-43f2-bb65-5a967629e238 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:38.636557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da754825-4658-4c79-8395-acf10b152489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:38.636557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da754825-4658-4c79-8395-acf10b152489 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:38.636557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da754825-4658-4c79-8395-acf10b152489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:38.636557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da754825-4658-4c79-8395-acf10b152489 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:38.636880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da754825-4658-4c79-8395-acf10b152489 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:38.644086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da754825-4658-4c79-8395-acf10b152489 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:38.800870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 878/1764] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:38 2022] POST /identity/v3/projects => generated 399 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:38.805183 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d1933-154c-4bb8-954a-47f350df7136 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:38.805543 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d1933-154c-4bb8-954a-47f350df7136 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:38.805896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d1933-154c-4bb8-954a-47f350df7136 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:38.807571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 879/1765] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 23 11:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:44:38.808357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d1933-154c-4bb8-954a-47f350df7136 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:38.874323 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 887/1766] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:38 2022] POST /identity/v3/users => generated 288 bytes in 493 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:38.874323 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f864053-187c-46de-95d3-90abb683312a None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:38.874822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f864053-187c-46de-95d3-90abb683312a None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:38.875113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f864053-187c-46de-95d3-90abb683312a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:39.007674 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d1933-154c-4bb8-954a-47f350df7136 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:39.009834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-8f864053-187c-46de-95d3-90abb683312a None None] MFA Rules not processed for user `161c5ed1e47e450290a1f428ee100b68`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:39.012875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-8f864053-187c-46de-95d3-90abb683312a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:39.022235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b47c0-3a8d-4c29-816f-3ddac32e1def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:39.024029 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b47c0-3a8d-4c29-816f-3ddac32e1def admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:39.024314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b47c0-3a8d-4c29-816f-3ddac32e1def admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:39.024588 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b47c0-3a8d-4c29-816f-3ddac32e1def admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:39.029574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b47c0-3a8d-4c29-816f-3ddac32e1def admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:39.035258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b47c0-3a8d-4c29-816f-3ddac32e1def admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:39.071648 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:39.153002 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 880/1767] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:38 2022] POST /identity/v3/users => generated 306 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:39.154017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b47c0-3a8d-4c29-816f-3ddac32e1def admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:39.175162 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b47c0-3a8d-4c29-816f-3ddac32e1def admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:39.188858 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f416ac2-e9ed-45e0-b22f-1ee11148ee88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:39.191943 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f416ac2-e9ed-45e0-b22f-1ee11148ee88 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:39.191943 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f416ac2-e9ed-45e0-b22f-1ee11148ee88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:39.191943 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f416ac2-e9ed-45e0-b22f-1ee11148ee88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:39.191943 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f416ac2-e9ed-45e0-b22f-1ee11148ee88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:39.198519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f416ac2-e9ed-45e0-b22f-1ee11148ee88 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:39.209942 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 888/1768] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 23 11:44:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:39.210857 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f864053-187c-46de-95d3-90abb683312a None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:39.228831 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 881/1769] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:39.231722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f416ac2-e9ed-45e0-b22f-1ee11148ee88 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:39.251950 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f864053-187c-46de-95d3-90abb683312a None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:39.257498 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d5a62-60ba-4967-82bd-fe0d3a7c5bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:39.258599 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d5a62-60ba-4967-82bd-fe0d3a7c5bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:39.259391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d5a62-60ba-4967-82bd-fe0d3a7c5bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:39.259391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d5a62-60ba-4967-82bd-fe0d3a7c5bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:39.260670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d5a62-60ba-4967-82bd-fe0d3a7c5bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:39.261284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f416ac2-e9ed-45e0-b22f-1ee11148ee88 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:39.263498 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d5a62-60ba-4967-82bd-fe0d3a7c5bc5 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:39.276876 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 889/1770] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:39.279550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d5a62-60ba-4967-82bd-fe0d3a7c5bc5 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:39.281337 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-1a0d5a62-60ba-4967-82bd-fe0d3a7c5bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:39.283781 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc6327-e469-42bb-ade9-fc5ffe280eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:39.283781 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc6327-e469-42bb-ade9-fc5ffe280eab admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:39.283781 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc6327-e469-42bb-ade9-fc5ffe280eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:39.283781 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc6327-e469-42bb-ade9-fc5ffe280eab admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:39.284141 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fc6327-e469-42bb-ade9-fc5ffe280eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:39.287166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fc6327-e469-42bb-ade9-fc5ffe280eab admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:39.319626 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 882/1771] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:39.320193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc6327-e469-42bb-ade9-fc5ffe280eab admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:39.351225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc6327-e469-42bb-ade9-fc5ffe280eab admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:39.356395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d5a62-60ba-4967-82bd-fe0d3a7c5bc5 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:39.365191 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260eaf9-0a11-48d6-bf3f-d53dd0b1611c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:39.366914 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260eaf9-0a11-48d6-bf3f-d53dd0b1611c admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:39.367348 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260eaf9-0a11-48d6-bf3f-d53dd0b1611c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:39.375351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260eaf9-0a11-48d6-bf3f-d53dd0b1611c admin admin] PATH_INFO: `/v3/projects/3eb3e636461c4a319cc12584cc931158/users/fb7ae41a0dab4776aa738fe480c55099/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:39.384257 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc80586-f3ae-4228-b0cf-0984de16c6ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:39.384944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc80586-f3ae-4228-b0cf-0984de16c6ed service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:39.385245 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc80586-f3ae-4228-b0cf-0984de16c6ed service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:39.385563 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc80586-f3ae-4228-b0cf-0984de16c6ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:39.403456 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc80586-f3ae-4228-b0cf-0984de16c6ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:39.405854 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc80586-f3ae-4228-b0cf-0984de16c6ed service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:39.420759 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8260eaf9-0a11-48d6-bf3f-d53dd0b1611c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eb3e636461c4a319cc12584cc931158, user_id=fb7ae41a0dab4776aa738fe480c55099, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:39.433711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8260eaf9-0a11-48d6-bf3f-d53dd0b1611c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:39.465693 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 890/1772] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:39.467549 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc80586-f3ae-4228-b0cf-0984de16c6ed service nova] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:39.497338 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc80586-f3ae-4228-b0cf-0984de16c6ed service nova] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:39.506712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228669f3-47e3-4d8e-be3c-2c9163c04c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:39.507629 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228669f3-47e3-4d8e-be3c-2c9163c04c20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:39.510717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228669f3-47e3-4d8e-be3c-2c9163c04c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:39.510717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228669f3-47e3-4d8e-be3c-2c9163c04c20 admin admin] PATH_INFO: `/v3/projects/8333f0921023477fa9470a40e3cca03c/users/7b26b318fbef4edfb7e1caa5156e1bfc/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:39.541798 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228669f3-47e3-4d8e-be3c-2c9163c04c20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8333f0921023477fa9470a40e3cca03c, user_id=7b26b318fbef4edfb7e1caa5156e1bfc, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:39.544305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228669f3-47e3-4d8e-be3c-2c9163c04c20 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:39.703349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 883/1773] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:39 2022] PUT /identity/v3/projects/3eb3e636461c4a319cc12584cc931158/users/fb7ae41a0dab4776aa738fe480c55099/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:39.708543 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260eaf9-0a11-48d6-bf3f-d53dd0b1611c admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:39.723286 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 891/1774] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:39 2022] PUT /identity/v3/projects/8333f0921023477fa9470a40e3cca03c/users/7b26b318fbef4edfb7e1caa5156e1bfc/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:39.726930 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aaff0-5a8e-4806-9e30-9326f67d3c72 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:39.727740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aaff0-5a8e-4806-9e30-9326f67d3c72 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:39.728451 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aaff0-5a8e-4806-9e30-9326f67d3c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:39.767264 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c67aaff0-5a8e-4806-9e30-9326f67d3c72 None None] MFA Rules not processed for user `7b26b318fbef4edfb7e1caa5156e1bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:39.768549 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c67aaff0-5a8e-4806-9e30-9326f67d3c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:39.905164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:39.944610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260eaf9-0a11-48d6-bf3f-d53dd0b1611c admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:39.949816 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900cf0e3-e36d-4cf0-b75d-94d88c2953c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:39.951499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900cf0e3-e36d-4cf0-b75d-94d88c2953c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:39.951977 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900cf0e3-e36d-4cf0-b75d-94d88c2953c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:39.952478 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900cf0e3-e36d-4cf0-b75d-94d88c2953c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:39.953890 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900cf0e3-e36d-4cf0-b75d-94d88c2953c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:39.956159 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900cf0e3-e36d-4cf0-b75d-94d88c2953c3 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:39.960396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 892/1775] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:39.978433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 884/1776] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:39.979019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67aaff0-5a8e-4806-9e30-9326f67d3c72 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:39.980080 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900cf0e3-e36d-4cf0-b75d-94d88c2953c3 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:40.003649 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67aaff0-5a8e-4806-9e30-9326f67d3c72 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:40.009976 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900cf0e3-e36d-4cf0-b75d-94d88c2953c3 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:40.012744 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83732222-a1e6-49b4-bd89-41cefee92e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:40.014140 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83732222-a1e6-49b4-bd89-41cefee92e9b admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:40.015324 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83732222-a1e6-49b4-bd89-41cefee92e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:40.016436 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83732222-a1e6-49b4-bd89-41cefee92e9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:40.017364 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83732222-a1e6-49b4-bd89-41cefee92e9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:40.019714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83732222-a1e6-49b4-bd89-41cefee92e9b admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:40.023135 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c024ef-4e7b-4fc2-84bb-7ca39c16ae4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:40.024447 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c024ef-4e7b-4fc2-84bb-7ca39c16ae4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:40.025135 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c024ef-4e7b-4fc2-84bb-7ca39c16ae4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:40.025687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c024ef-4e7b-4fc2-84bb-7ca39c16ae4e admin admin] PATH_INFO: `/v3/projects/3eb3e636461c4a319cc12584cc931158/users/fb7ae41a0dab4776aa738fe480c55099/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:40.030712 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c024ef-4e7b-4fc2-84bb-7ca39c16ae4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eb3e636461c4a319cc12584cc931158, user_id=fb7ae41a0dab4776aa738fe480c55099, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:40.037108 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c024ef-4e7b-4fc2-84bb-7ca39c16ae4e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:40.058637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 885/1777] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:39 2022] PUT /identity/v3/projects/3eb3e636461c4a319cc12584cc931158/users/fb7ae41a0dab4776aa738fe480c55099/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:40.070198 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbebc53-b13e-47c9-9986-397d932b5e46 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:40.071048 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbebc53-b13e-47c9-9986-397d932b5e46 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:40.071950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbebc53-b13e-47c9-9986-397d932b5e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:40.130599 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 893/1778] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:39 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:40.132624 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83732222-a1e6-49b4-bd89-41cefee92e9b admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:40.159825 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-2fbebc53-b13e-47c9-9986-397d932b5e46 None None] MFA Rules not processed for user `fb7ae41a0dab4776aa738fe480c55099`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:40.166630 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2fbebc53-b13e-47c9-9986-397d932b5e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:40.254782 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:40.313266 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83732222-a1e6-49b4-bd89-41cefee92e9b admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:40.330492 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a14d3b-01fb-44f9-bb3a-1ceaea3f009b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:40.331558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a14d3b-01fb-44f9-bb3a-1ceaea3f009b admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:40.331855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a14d3b-01fb-44f9-bb3a-1ceaea3f009b admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:40.334501 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a14d3b-01fb-44f9-bb3a-1ceaea3f009b admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:40.335291 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a14d3b-01fb-44f9-bb3a-1ceaea3f009b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:40.339861 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a14d3b-01fb-44f9-bb3a-1ceaea3f009b admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:40.376712 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 886/1779] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:40 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:40.397921 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbebc53-b13e-47c9-9986-397d932b5e46 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:40.411718 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2fbebc53-b13e-47c9-9986-397d932b5e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:40.467307 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 894/1780] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:40 2022] POST /identity/v3/users => generated 294 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:40.468180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a14d3b-01fb-44f9-bb3a-1ceaea3f009b admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:40.471866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d1a14d3b-01fb-44f9-bb3a-1ceaea3f009b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:40.516783 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a14d3b-01fb-44f9-bb3a-1ceaea3f009b admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:40.534174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3e34e-d223-441d-9de0-15d5bcfc0f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:40.535346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3e34e-d223-441d-9de0-15d5bcfc0f53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:40.535935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3e34e-d223-441d-9de0-15d5bcfc0f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:40.536364 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3e34e-d223-441d-9de0-15d5bcfc0f53 admin admin] PATH_INFO: `/v3/projects/3b4d7c88bdb94f9598daf42f9e37924a` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:40.539058 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3e34e-d223-441d-9de0-15d5bcfc0f53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b4d7c88bdb94f9598daf42f9e37924a)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:40.542281 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3e34e-d223-441d-9de0-15d5bcfc0f53 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:40.574870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbebc53-b13e-47c9-9986-397d932b5e46 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:40.589313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f294dbb-b97f-4c1b-bb1e-d3c26ba2a609 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:40.590158 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f294dbb-b97f-4c1b-bb1e-d3c26ba2a609 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:40.590559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f294dbb-b97f-4c1b-bb1e-d3c26ba2a609 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:40.590961 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f294dbb-b97f-4c1b-bb1e-d3c26ba2a609 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:40.614213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f294dbb-b97f-4c1b-bb1e-d3c26ba2a609 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:40.615912 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f294dbb-b97f-4c1b-bb1e-d3c26ba2a609 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:40.649236 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 887/1781] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:40 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:44:40.650110 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f294dbb-b97f-4c1b-bb1e-d3c26ba2a609 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:40.668877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f294dbb-b97f-4c1b-bb1e-d3c26ba2a609 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:40.686085 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53753f30-a7e8-4c87-86cf-2578f305af5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:40.688724 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53753f30-a7e8-4c87-86cf-2578f305af5a admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:40.689015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53753f30-a7e8-4c87-86cf-2578f305af5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:40.689305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53753f30-a7e8-4c87-86cf-2578f305af5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:40.690045 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53753f30-a7e8-4c87-86cf-2578f305af5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:40.696225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53753f30-a7e8-4c87-86cf-2578f305af5a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:40.744776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 888/1782] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:40.749322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53753f30-a7e8-4c87-86cf-2578f305af5a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:40.763822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-82c3e34e-d223-441d-9de0-15d5bcfc0f53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b4d7c88bdb94f9598daf42f9e37924a', 'request_id': 'req-82c3e34e-d223-441d-9de0-15d5bcfc0f53'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:40.782272 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53753f30-a7e8-4c87-86cf-2578f305af5a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:40.788376 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-82c3e34e-d223-441d-9de0-15d5bcfc0f53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b4d7c88bdb94f9598daf42f9e37924a', 'request_id': 'req-82c3e34e-d223-441d-9de0-15d5bcfc0f53'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:40.839562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:40.918593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-82c3e34e-d223-441d-9de0-15d5bcfc0f53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b4d7c88bdb94f9598daf42f9e37924a', 'request_id': 'req-82c3e34e-d223-441d-9de0-15d5bcfc0f53'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:40.933938 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc109f-c44a-41e3-a94a-44bc6969f91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:40.935585 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-82c3e34e-d223-441d-9de0-15d5bcfc0f53 admin admin] The token cache is being invalidate because project 3b4d7c88bdb94f9598daf42f9e37924a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:40.941952 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-82c3e34e-d223-441d-9de0-15d5bcfc0f53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:40.942689 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc109f-c44a-41e3-a94a-44bc6969f91e admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:40.942689 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc109f-c44a-41e3-a94a-44bc6969f91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:40.942847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-82c3e34e-d223-441d-9de0-15d5bcfc0f53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:40.942915 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc109f-c44a-41e3-a94a-44bc6969f91e admin admin] PATH_INFO: `/v3/projects/823eb0bf1d4e4afc876da6311f40ca55/users/68c62a4b4a9845d9bdc10f9f8afc7a13/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:40.943030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 895/1783] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:40 2022] DELETE /identity/v3/projects/3b4d7c88bdb94f9598daf42f9e37924a => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:40.953882 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c4243-f40b-4b51-8ed4-78afd71c4cf8 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:40.956394 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c4243-f40b-4b51-8ed4-78afd71c4cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:40.957477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c4243-f40b-4b51-8ed4-78afd71c4cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:40.967693 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc109f-c44a-41e3-a94a-44bc6969f91e admin admin] RBAC: Authorizing `identity:create_grant(project_id=823eb0bf1d4e4afc876da6311f40ca55, user_id=68c62a4b4a9845d9bdc10f9f8afc7a13, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:40.970940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc109f-c44a-41e3-a94a-44bc6969f91e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:40.986060 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:41.002955 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-489c4243-f40b-4b51-8ed4-78afd71c4cf8 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:41.005196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-489c4243-f40b-4b51-8ed4-78afd71c4cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:41.016972 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 896/1784] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:41.025011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a616bb-0672-478f-b24e-84df6174441f None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:41.025295 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a616bb-0672-478f-b24e-84df6174441f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:41.025575 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a616bb-0672-478f-b24e-84df6174441f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:41.073512 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 889/1785] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:40 2022] PUT /identity/v3/projects/823eb0bf1d4e4afc876da6311f40ca55/users/68c62a4b4a9845d9bdc10f9f8afc7a13/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:41.080470 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-92a616bb-0672-478f-b24e-84df6174441f None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:41.082326 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-92a616bb-0672-478f-b24e-84df6174441f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:41.091053 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc109f-c44a-41e3-a94a-44bc6969f91e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:41.107523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c4bc109f-c44a-41e3-a94a-44bc6969f91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:41.258089 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 897/1786] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:44:41.258940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc109f-c44a-41e3-a94a-44bc6969f91e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:41.260805 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a616bb-0672-478f-b24e-84df6174441f None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:41.263727 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-92a616bb-0672-478f-b24e-84df6174441f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:41.271335 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ff2c4-e786-4640-81ae-de6ca6006fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:41.272826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ff2c4-e786-4640-81ae-de6ca6006fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:41.273327 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ff2c4-e786-4640-81ae-de6ca6006fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:41.279658 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ff2c4-e786-4640-81ae-de6ca6006fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:41.283569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ff2c4-e786-4640-81ae-de6ca6006fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:41.283569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ff2c4-e786-4640-81ae-de6ca6006fe7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:41.322049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 890/1787] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:41.325133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ff2c4-e786-4640-81ae-de6ca6006fe7 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:41.355783 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a616bb-0672-478f-b24e-84df6174441f None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:41.358515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ff2c4-e786-4640-81ae-de6ca6006fe7 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:41.364928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91029883-96cf-4324-a829-224ce02d8c24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:41.368558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91029883-96cf-4324-a829-224ce02d8c24 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:41.369333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91029883-96cf-4324-a829-224ce02d8c24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:41.369527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61449f-f66e-4d36-8183-eac9e208fa9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:41.370118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91029883-96cf-4324-a829-224ce02d8c24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:41.370455 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61449f-f66e-4d36-8183-eac9e208fa9e admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:41.370889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61449f-f66e-4d36-8183-eac9e208fa9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:41.371293 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61449f-f66e-4d36-8183-eac9e208fa9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:41.372158 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d61449f-f66e-4d36-8183-eac9e208fa9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:41.372678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d61449f-f66e-4d36-8183-eac9e208fa9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:41.374430 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d61449f-f66e-4d36-8183-eac9e208fa9e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:41.380873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-91029883-96cf-4324-a829-224ce02d8c24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:41.415497 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 891/1788] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:44:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:44:41.415497 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61449f-f66e-4d36-8183-eac9e208fa9e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:41.433060 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61449f-f66e-4d36-8183-eac9e208fa9e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:41.443967 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38bfbe-4d78-49d7-b5f4-58e5274ab8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:41.446010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38bfbe-4d78-49d7-b5f4-58e5274ab8e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:41.446404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38bfbe-4d78-49d7-b5f4-58e5274ab8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:41.446784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38bfbe-4d78-49d7-b5f4-58e5274ab8e0 admin admin] PATH_INFO: `/v3/projects/823eb0bf1d4e4afc876da6311f40ca55/users/68c62a4b4a9845d9bdc10f9f8afc7a13/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:41.449061 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a38bfbe-4d78-49d7-b5f4-58e5274ab8e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=823eb0bf1d4e4afc876da6311f40ca55, user_id=68c62a4b4a9845d9bdc10f9f8afc7a13, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:41.453091 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a38bfbe-4d78-49d7-b5f4-58e5274ab8e0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:41.471049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 892/1789] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:41 2022] PUT /identity/v3/projects/823eb0bf1d4e4afc876da6311f40ca55/users/68c62a4b4a9845d9bdc10f9f8afc7a13/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:41.472244 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38bfbe-4d78-49d7-b5f4-58e5274ab8e0 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:41.578128 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91029883-96cf-4324-a829-224ce02d8c24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:41.580325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91029883-96cf-4324-a829-224ce02d8c24 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:41.611715 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 898/1790] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:41 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:41.614158 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5844943-adcc-4bb0-b924-5ffa1e6713e2 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:41.614577 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5844943-adcc-4bb0-b924-5ffa1e6713e2 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:41.614857 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5844943-adcc-4bb0-b924-5ffa1e6713e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:41.657938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-b5844943-adcc-4bb0-b924-5ffa1e6713e2 None None] MFA Rules not processed for user `68c62a4b4a9845d9bdc10f9f8afc7a13`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:41.659956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b5844943-adcc-4bb0-b924-5ffa1e6713e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:41.671592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38bfbe-4d78-49d7-b5f4-58e5274ab8e0 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:41.689890 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb97b4b-23a4-452c-98c6-c9c4c330808e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:41.695225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb97b4b-23a4-452c-98c6-c9c4c330808e admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:41.695521 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb97b4b-23a4-452c-98c6-c9c4c330808e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:41.695841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb97b4b-23a4-452c-98c6-c9c4c330808e admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:41.696854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb97b4b-23a4-452c-98c6-c9c4c330808e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:41.699768 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb97b4b-23a4-452c-98c6-c9c4c330808e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:41.783166 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:41.831659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 893/1791] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:41 2022] POST /identity/v3/projects => generated 377 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:41.832827 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb97b4b-23a4-452c-98c6-c9c4c330808e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:41.866103 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 899/1792] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:41 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:41.877259 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5844943-adcc-4bb0-b924-5ffa1e6713e2 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:41.880402 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b5844943-adcc-4bb0-b924-5ffa1e6713e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:41.988782 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb97b4b-23a4-452c-98c6-c9c4c330808e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:42.001140 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5844943-adcc-4bb0-b924-5ffa1e6713e2 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:42.005057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5d837-ff48-46ef-8bd6-7f42120d34a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:42.005805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5d837-ff48-46ef-8bd6-7f42120d34a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:42.006127 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5d837-ff48-46ef-8bd6-7f42120d34a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:42.006483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5d837-ff48-46ef-8bd6-7f42120d34a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:42.007312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5d837-ff48-46ef-8bd6-7f42120d34a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:42.011465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5d837-ff48-46ef-8bd6-7f42120d34a9 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:42.062474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:42.073224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8ad15-68ae-47c6-b8b1-04cdce33b418 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:42.087813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8ad15-68ae-47c6-b8b1-04cdce33b418 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:42.087813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8ad15-68ae-47c6-b8b1-04cdce33b418 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:42.087813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8ad15-68ae-47c6-b8b1-04cdce33b418 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:42.087813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-0af8ad15-68ae-47c6-b8b1-04cdce33b418 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:42.158768 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8ad15-68ae-47c6-b8b1-04cdce33b418 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:42.160342 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8ad15-68ae-47c6-b8b1-04cdce33b418 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:42.180360 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 900/1793] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:44:42.327035 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 894/1794] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:41 2022] POST /identity/v3/users => generated 289 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:42.327620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8ad15-68ae-47c6-b8b1-04cdce33b418 service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:42.327620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8ad15-68ae-47c6-b8b1-04cdce33b418 service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:42.332907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bf164-905c-4c1d-9cb8-dbf12080934f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:42.334253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bf164-905c-4c1d-9cb8-dbf12080934f admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:42.334253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bf164-905c-4c1d-9cb8-dbf12080934f admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:42.334710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bf164-905c-4c1d-9cb8-dbf12080934f admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:42.335549 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063bf164-905c-4c1d-9cb8-dbf12080934f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:42.355119 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063bf164-905c-4c1d-9cb8-dbf12080934f admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:42.373861 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 901/1795] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:42.377631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5d837-ff48-46ef-8bd6-7f42120d34a9 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:42.388235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5d837-ff48-46ef-8bd6-7f42120d34a9 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:42.393173 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a08346-89ee-4140-898a-41035b895bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:42.393848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a08346-89ee-4140-898a-41035b895bec admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:42.394132 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a08346-89ee-4140-898a-41035b895bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:42.394408 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a08346-89ee-4140-898a-41035b895bec admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:42.395323 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a08346-89ee-4140-898a-41035b895bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:42.396960 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a08346-89ee-4140-898a-41035b895bec admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:42.410669 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 895/1796] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:44:42.413705 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bf164-905c-4c1d-9cb8-dbf12080934f admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:42.425840 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bf164-905c-4c1d-9cb8-dbf12080934f admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:42.432478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d20f1d-d754-4e63-afa1-c36105f32e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:42.434221 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d20f1d-d754-4e63-afa1-c36105f32e04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:42.434496 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d20f1d-d754-4e63-afa1-c36105f32e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:42.435659 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d20f1d-d754-4e63-afa1-c36105f32e04 admin admin] PATH_INFO: `/v3/projects/ad2e8f4a478644e79b8fe7bf4262e6e5/users/5a042871c802416f92ea6cf9ae925f67/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:42.450277 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d20f1d-d754-4e63-afa1-c36105f32e04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad2e8f4a478644e79b8fe7bf4262e6e5, user_id=5a042871c802416f92ea6cf9ae925f67, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:42.453088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d20f1d-d754-4e63-afa1-c36105f32e04 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:42.587042 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 902/1797] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:42 2022] PUT /identity/v3/projects/ad2e8f4a478644e79b8fe7bf4262e6e5/users/5a042871c802416f92ea6cf9ae925f67/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:42.598072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f170aa-3479-4f4f-8139-9d0539099f94 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:42.598364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f170aa-3479-4f4f-8139-9d0539099f94 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:42.598684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f170aa-3479-4f4f-8139-9d0539099f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:42.626468 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-e9f170aa-3479-4f4f-8139-9d0539099f94 None None] MFA Rules not processed for user `5a042871c802416f92ea6cf9ae925f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:42.627757 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e9f170aa-3479-4f4f-8139-9d0539099f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:42.694523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 896/1798] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:42.699022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d20f1d-d754-4e63-afa1-c36105f32e04 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:42.769385 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d20f1d-d754-4e63-afa1-c36105f32e04 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:42.774285 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e7e92-57cf-41b6-8b2e-2691f6110ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:42.774981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e7e92-57cf-41b6-8b2e-2691f6110ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:42.775267 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e7e92-57cf-41b6-8b2e-2691f6110ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:42.775542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e7e92-57cf-41b6-8b2e-2691f6110ab5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:42.776379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e7e92-57cf-41b6-8b2e-2691f6110ab5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:42.778442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e7e92-57cf-41b6-8b2e-2691f6110ab5 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:42.921458 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 903/1799] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:42 2022] POST /identity/v3/projects => generated 377 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:42.922801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f170aa-3479-4f4f-8139-9d0539099f94 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:42.934191 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f170aa-3479-4f4f-8139-9d0539099f94 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:43.020140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016855c-5a48-46b6-89af-72cf41486b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:43.023350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016855c-5a48-46b6-89af-72cf41486b3c admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:43.023710 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016855c-5a48-46b6-89af-72cf41486b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:43.024117 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016855c-5a48-46b6-89af-72cf41486b3c admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:43.024907 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016855c-5a48-46b6-89af-72cf41486b3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:43.027137 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016855c-5a48-46b6-89af-72cf41486b3c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:43.155166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 897/1800] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:42 2022] POST /identity/v3/users => generated 289 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:43.157234 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e7e92-57cf-41b6-8b2e-2691f6110ab5 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:43.169069 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e7e92-57cf-41b6-8b2e-2691f6110ab5 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:43.176809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd81bc2-a30b-4d40-bf55-bf0b88cae75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:43.176809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd81bc2-a30b-4d40-bf55-bf0b88cae75b admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:43.176809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd81bc2-a30b-4d40-bf55-bf0b88cae75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:43.176809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd81bc2-a30b-4d40-bf55-bf0b88cae75b admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:43.177198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd81bc2-a30b-4d40-bf55-bf0b88cae75b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:43.178929 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd81bc2-a30b-4d40-bf55-bf0b88cae75b admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:43.195605 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 904/1801] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:43.196961 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016855c-5a48-46b6-89af-72cf41486b3c admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:43.209208 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016855c-5a48-46b6-89af-72cf41486b3c admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:43.220596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26600f43-564a-4072-8a5e-d2ab5b8bb71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:43.221302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26600f43-564a-4072-8a5e-d2ab5b8bb71f admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:43.221583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26600f43-564a-4072-8a5e-d2ab5b8bb71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:43.221854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26600f43-564a-4072-8a5e-d2ab5b8bb71f admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:43.222629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26600f43-564a-4072-8a5e-d2ab5b8bb71f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:43.224510 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26600f43-564a-4072-8a5e-d2ab5b8bb71f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:43.236104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 898/1802] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:43.241023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd81bc2-a30b-4d40-bf55-bf0b88cae75b admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:43.252152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd81bc2-a30b-4d40-bf55-bf0b88cae75b admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:43.261223 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495455d-df74-4b24-8bbc-dbd956ad7afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:43.262201 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495455d-df74-4b24-8bbc-dbd956ad7afe admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:43.262573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495455d-df74-4b24-8bbc-dbd956ad7afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:43.263063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495455d-df74-4b24-8bbc-dbd956ad7afe admin admin] PATH_INFO: `/v3/projects/bc47f1c70370473b9dc9e23ccb81534e/users/8c041acec9cc49adabb871298922116c/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:43.286699 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495455d-df74-4b24-8bbc-dbd956ad7afe admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc47f1c70370473b9dc9e23ccb81534e, user_id=8c041acec9cc49adabb871298922116c, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:43.286699 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495455d-df74-4b24-8bbc-dbd956ad7afe admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:43.378162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 905/1803] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:43 2022] PUT /identity/v3/projects/bc47f1c70370473b9dc9e23ccb81534e/users/8c041acec9cc49adabb871298922116c/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:43.388909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd7d48-66d8-4475-b245-4b6acb197d6a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:43.389885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd7d48-66d8-4475-b245-4b6acb197d6a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:43.390193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd7d48-66d8-4475-b245-4b6acb197d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:43.419017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-0ddd7d48-66d8-4475-b245-4b6acb197d6a None None] MFA Rules not processed for user `8c041acec9cc49adabb871298922116c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:43.420342 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-0ddd7d48-66d8-4475-b245-4b6acb197d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:43.502453 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 899/1804] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:43.507257 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495455d-df74-4b24-8bbc-dbd956ad7afe admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:43.596691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495455d-df74-4b24-8bbc-dbd956ad7afe admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:43.603769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975813b-55dd-49bc-984d-e5d77330dd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:43.604442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975813b-55dd-49bc-984d-e5d77330dd61 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:43.604728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975813b-55dd-49bc-984d-e5d77330dd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:43.605007 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975813b-55dd-49bc-984d-e5d77330dd61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:43.605715 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975813b-55dd-49bc-984d-e5d77330dd61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:43.608226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975813b-55dd-49bc-984d-e5d77330dd61 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:43.775377 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 906/1805] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:43 2022] POST /identity/v3/projects => generated 379 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:43.776563 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd7d48-66d8-4475-b245-4b6acb197d6a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:43.787887 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd7d48-66d8-4475-b245-4b6acb197d6a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:43.849773 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d69352-a200-457d-91c0-0c507318fcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:43.850602 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d69352-a200-457d-91c0-0c507318fcdd admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:43.851126 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d69352-a200-457d-91c0-0c507318fcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:43.851500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d69352-a200-457d-91c0-0c507318fcdd admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:43.852336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d69352-a200-457d-91c0-0c507318fcdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:43.854613 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d69352-a200-457d-91c0-0c507318fcdd admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:43.952564 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 900/1806] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:43 2022] POST /identity/v3/users => generated 296 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:43.956670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975813b-55dd-49bc-984d-e5d77330dd61 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:43.967427 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975813b-55dd-49bc-984d-e5d77330dd61 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:43.972836 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdcc69f-6b70-429c-95d5-29cfff4fc38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:43.973505 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcc69f-6b70-429c-95d5-29cfff4fc38e admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:43.973785 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcc69f-6b70-429c-95d5-29cfff4fc38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:43.974077 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcc69f-6b70-429c-95d5-29cfff4fc38e admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:43.974844 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdcc69f-6b70-429c-95d5-29cfff4fc38e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:43.976444 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdcc69f-6b70-429c-95d5-29cfff4fc38e admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:43.990736 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 907/1807] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:43.991856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d69352-a200-457d-91c0-0c507318fcdd admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:44.003151 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d69352-a200-457d-91c0-0c507318fcdd admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:44.007848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfed0a5-0f96-4d97-ac7b-08a509ba440f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:44.008768 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfed0a5-0f96-4d97-ac7b-08a509ba440f admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:44.009049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfed0a5-0f96-4d97-ac7b-08a509ba440f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:44.009323 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfed0a5-0f96-4d97-ac7b-08a509ba440f admin admin] PATH_INFO: `/v3/projects/3b9503205d1b4ca7a113225c76ddda4b/users/e7d84edcfbfb42b6a8e580a99da4feea/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:44.021537 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:44.028770 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfed0a5-0f96-4d97-ac7b-08a509ba440f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b9503205d1b4ca7a113225c76ddda4b, user_id=e7d84edcfbfb42b6a8e580a99da4feea, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:44.032247 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfed0a5-0f96-4d97-ac7b-08a509ba440f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:44.131094 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 901/1808] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:43 2022] PUT /identity/v3/projects/3b9503205d1b4ca7a113225c76ddda4b/users/e7d84edcfbfb42b6a8e580a99da4feea/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:44.135178 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdcc69f-6b70-429c-95d5-29cfff4fc38e admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:44.145816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdcc69f-6b70-429c-95d5-29cfff4fc38e admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:44.208618 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71788a-6202-4f3a-9824-59397261398b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:44.218219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71788a-6202-4f3a-9824-59397261398b admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:44.218219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71788a-6202-4f3a-9824-59397261398b admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:44.218219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71788a-6202-4f3a-9824-59397261398b admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:44.218219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71788a-6202-4f3a-9824-59397261398b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:44.218219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71788a-6202-4f3a-9824-59397261398b admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:44.223629 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 908/1809] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:44.228604 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfed0a5-0f96-4d97-ac7b-08a509ba440f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:44.239982 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfed0a5-0f96-4d97-ac7b-08a509ba440f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:44.242845 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:44.260535 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485fbb5-7685-4244-843c-993a9533d8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:44.260535 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485fbb5-7685-4244-843c-993a9533d8d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:44.260535 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485fbb5-7685-4244-843c-993a9533d8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:44.260535 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485fbb5-7685-4244-843c-993a9533d8d8 admin admin] PATH_INFO: `/v3/projects/3b9503205d1b4ca7a113225c76ddda4b/users/e7d84edcfbfb42b6a8e580a99da4feea/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:44.260889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485fbb5-7685-4244-843c-993a9533d8d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b9503205d1b4ca7a113225c76ddda4b, user_id=e7d84edcfbfb42b6a8e580a99da4feea, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:44.260889 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485fbb5-7685-4244-843c-993a9533d8d8 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:44.265108 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 902/1810] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:44 2022] PUT /identity/v3/projects/3b9503205d1b4ca7a113225c76ddda4b/users/e7d84edcfbfb42b6a8e580a99da4feea/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:44.275239 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c53b9f-7daf-4306-a30b-4cf7c36ef987 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:44.275516 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c53b9f-7daf-4306-a30b-4cf7c36ef987 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:44.275988 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c53b9f-7daf-4306-a30b-4cf7c36ef987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:44.303341 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-59c53b9f-7daf-4306-a30b-4cf7c36ef987 None None] MFA Rules not processed for user `e7d84edcfbfb42b6a8e580a99da4feea`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:44.304560 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-59c53b9f-7daf-4306-a30b-4cf7c36ef987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:44.375062 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 909/1811] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:44.386255 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485fbb5-7685-4244-843c-993a9533d8d8 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:44.460034 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485fbb5-7685-4244-843c-993a9533d8d8 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:44.465980 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa04697-996c-4542-8458-2ab38760e925 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:44.467598 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa04697-996c-4542-8458-2ab38760e925 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:44.468010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa04697-996c-4542-8458-2ab38760e925 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:44.468392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa04697-996c-4542-8458-2ab38760e925 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:44.480173 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa04697-996c-4542-8458-2ab38760e925 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:44.482989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa04697-996c-4542-8458-2ab38760e925 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:44.499968 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 903/1812] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:44 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:44.668639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c53b9f-7daf-4306-a30b-4cf7c36ef987 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:44.678980 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c53b9f-7daf-4306-a30b-4cf7c36ef987 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:44.682638 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e143c-7f88-4fc6-adbf-85cfe7931f6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:44.683261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e143c-7f88-4fc6-adbf-85cfe7931f6f service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:44.683545 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e143c-7f88-4fc6-adbf-85cfe7931f6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:44.683821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e143c-7f88-4fc6-adbf-85cfe7931f6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:44.766202 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e143c-7f88-4fc6-adbf-85cfe7931f6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:44.768013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e143c-7f88-4fc6-adbf-85cfe7931f6f service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:44.780638 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 910/1813] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:44.851361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:45.244594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:46.187487 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:46.590917 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:51.603239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa04697-996c-4542-8458-2ab38760e925 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:51.612582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa04697-996c-4542-8458-2ab38760e925 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:51.616818 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31eb85e-411d-4805-83d8-0547225622dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:51.617482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31eb85e-411d-4805-83d8-0547225622dd service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:51.617795 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31eb85e-411d-4805-83d8-0547225622dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:51.618075 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31eb85e-411d-4805-83d8-0547225622dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:51.683483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31eb85e-411d-4805-83d8-0547225622dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:51.684464 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31eb85e-411d-4805-83d8-0547225622dd service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:51.698723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 904/1814] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:51 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:52.511488 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e143c-7f88-4fc6-adbf-85cfe7931f6f service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:52.511488 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-df5e143c-7f88-4fc6-adbf-85cfe7931f6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:52.559481 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:52.578493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e340493-b845-42c6-b7e6-9d28f8499b3f None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:52.579147 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e340493-b845-42c6-b7e6-9d28f8499b3f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:52.579758 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e340493-b845-42c6-b7e6-9d28f8499b3f None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:52.581489 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 905/1815] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 23 11:44:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:44:52.586049 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e143c-7f88-4fc6-adbf-85cfe7931f6f service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:52.590985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b184aea-45f9-437f-83b5-740d35a5a6ba None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:52.591437 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b184aea-45f9-437f-83b5-740d35a5a6ba None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:52.591544 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b588d1-2e22-4d19-8c56-546a2c515d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:52.591636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b184aea-45f9-437f-83b5-740d35a5a6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:52.592341 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b588d1-2e22-4d19-8c56-546a2c515d5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:52.592699 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b588d1-2e22-4d19-8c56-546a2c515d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:52.593147 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b588d1-2e22-4d19-8c56-546a2c515d5c admin admin] PATH_INFO: `/v3/users/ae6348a5810c413d9bca62f4769a51e6` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:52.595658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b588d1-2e22-4d19-8c56-546a2c515d5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae6348a5810c413d9bca62f4769a51e6)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:52.600956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b588d1-2e22-4d19-8c56-546a2c515d5c admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:52.615358 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5b184aea-45f9-437f-83b5-740d35a5a6ba None None] MFA Rules not processed for user `161c5ed1e47e450290a1f428ee100b68`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:52.618396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5b184aea-45f9-437f-83b5-740d35a5a6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:52.696028 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 906/1816] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 23 11:44:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:44:52.709781 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b184aea-45f9-437f-83b5-740d35a5a6ba None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:52.711230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5b184aea-45f9-437f-83b5-740d35a5a6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:52.725255 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b184aea-45f9-437f-83b5-740d35a5a6ba None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:52.730867 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a9869-82e3-4073-a0d6-f2976209b943 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:52.731549 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a9869-82e3-4073-a0d6-f2976209b943 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:52.731838 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a9869-82e3-4073-a0d6-f2976209b943 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:52.732112 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a9869-82e3-4073-a0d6-f2976209b943 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:52.745445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a9869-82e3-4073-a0d6-f2976209b943 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:52.749365 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a9869-82e3-4073-a0d6-f2976209b943 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:52.758678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 907/1817] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:44:52.819413 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a9869-82e3-4073-a0d6-f2976209b943 service nova] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:52.831013 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a9869-82e3-4073-a0d6-f2976209b943 service nova] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:52.836128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51562578-e03c-42be-ac27-87b4acdd89da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:52.836991 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51562578-e03c-42be-ac27-87b4acdd89da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:52.840277 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51562578-e03c-42be-ac27-87b4acdd89da admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:52.840277 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51562578-e03c-42be-ac27-87b4acdd89da admin admin] PATH_INFO: `/v3/users/5a042871c802416f92ea6cf9ae925f67` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:52.840277 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51562578-e03c-42be-ac27-87b4acdd89da admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a042871c802416f92ea6cf9ae925f67)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:52.840686 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51562578-e03c-42be-ac27-87b4acdd89da admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:52.854593 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:52.908257 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-24b588d1-2e22-4d19-8c56-546a2c515d5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae6348a5810c413d9bca62f4769a51e6'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:52.915854 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-24b588d1-2e22-4d19-8c56-546a2c515d5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae6348a5810c413d9bca62f4769a51e6'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:52.917117 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-24b588d1-2e22-4d19-8c56-546a2c515d5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae6348a5810c413d9bca62f4769a51e6'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:52.917210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-24b588d1-2e22-4d19-8c56-546a2c515d5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae6348a5810c413d9bca62f4769a51e6'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:52.930216 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-24b588d1-2e22-4d19-8c56-546a2c515d5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae6348a5810c413d9bca62f4769a51e6'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:53.058432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 911/1818] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:52 2022] DELETE /identity/v3/users/ae6348a5810c413d9bca62f4769a51e6 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:53.060189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50167cb7-6eea-4142-8850-01fbe98670ec None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:53.060586 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50167cb7-6eea-4142-8850-01fbe98670ec None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:53.060888 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50167cb7-6eea-4142-8850-01fbe98670ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:53.081614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-50167cb7-6eea-4142-8850-01fbe98670ec None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:53.082872 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-50167cb7-6eea-4142-8850-01fbe98670ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:53.150047 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 912/1819] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:53.163755 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50167cb7-6eea-4142-8850-01fbe98670ec None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:53.165208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-50167cb7-6eea-4142-8850-01fbe98670ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:53.173002 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-51562578-e03c-42be-ac27-87b4acdd89da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a042871c802416f92ea6cf9ae925f67'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:53.181294 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-51562578-e03c-42be-ac27-87b4acdd89da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a042871c802416f92ea6cf9ae925f67'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:53.182269 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-51562578-e03c-42be-ac27-87b4acdd89da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a042871c802416f92ea6cf9ae925f67'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:53.182885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-51562578-e03c-42be-ac27-87b4acdd89da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a042871c802416f92ea6cf9ae925f67'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:53.199019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-51562578-e03c-42be-ac27-87b4acdd89da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a042871c802416f92ea6cf9ae925f67'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:53.243876 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50167cb7-6eea-4142-8850-01fbe98670ec None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:53.247987 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7ab98-0ee4-4e40-aad8-657a7f8fb4a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:53.248623 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7ab98-0ee4-4e40-aad8-657a7f8fb4a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:53.249051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7ab98-0ee4-4e40-aad8-657a7f8fb4a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:53.249336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7ab98-0ee4-4e40-aad8-657a7f8fb4a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:53.252054 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-92f7ab98-0ee4-4e40-aad8-657a7f8fb4a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:53.264630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7ab98-0ee4-4e40-aad8-657a7f8fb4a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:53.266073 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7ab98-0ee4-4e40-aad8-657a7f8fb4a7 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:53.275796 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 913/1820] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:44:53.302129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 908/1821] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:52 2022] DELETE /identity/v3/users/5a042871c802416f92ea6cf9ae925f67 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:53.306494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f57f02-5dab-423b-bbda-7623f425de90 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:53.306805 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f57f02-5dab-423b-bbda-7623f425de90 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:53.307590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f57f02-5dab-423b-bbda-7623f425de90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:53.313401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:53.326728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-07f57f02-5dab-423b-bbda-7623f425de90 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:53.327934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-07f57f02-5dab-423b-bbda-7623f425de90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:53.400835 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 914/1822] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:53.411356 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51562578-e03c-42be-ac27-87b4acdd89da admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:53.412763 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-51562578-e03c-42be-ac27-87b4acdd89da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:53.471703 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51562578-e03c-42be-ac27-87b4acdd89da admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:53.475490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90841bac-df2f-4716-9677-158ed4789ab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:53.476148 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90841bac-df2f-4716-9677-158ed4789ab5 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:53.476430 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90841bac-df2f-4716-9677-158ed4789ab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:53.476707 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90841bac-df2f-4716-9677-158ed4789ab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:53.488045 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90841bac-df2f-4716-9677-158ed4789ab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:53.489826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90841bac-df2f-4716-9677-158ed4789ab5 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:53.502426 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 909/1823] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:53.701970 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f57f02-5dab-423b-bbda-7623f425de90 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:53.703622 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-07f57f02-5dab-423b-bbda-7623f425de90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:53.724227 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f57f02-5dab-423b-bbda-7623f425de90 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:53.730327 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35aef5-cf32-4975-9d00-7dc708f296d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:53.731239 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35aef5-cf32-4975-9d00-7dc708f296d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:53.731560 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35aef5-cf32-4975-9d00-7dc708f296d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:53.731789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35aef5-cf32-4975-9d00-7dc708f296d4 admin admin] PATH_INFO: `/v3/projects/220b20a472ef4bae838624c1994faf38` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:53.733132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35aef5-cf32-4975-9d00-7dc708f296d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=220b20a472ef4bae838624c1994faf38)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:53.735714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35aef5-cf32-4975-9d00-7dc708f296d4 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:53.881483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-9a35aef5-cf32-4975-9d00-7dc708f296d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '220b20a472ef4bae838624c1994faf38', 'request_id': 'req-9a35aef5-cf32-4975-9d00-7dc708f296d4'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:53.888256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-9a35aef5-cf32-4975-9d00-7dc708f296d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '220b20a472ef4bae838624c1994faf38', 'request_id': 'req-9a35aef5-cf32-4975-9d00-7dc708f296d4'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:53.957752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90841bac-df2f-4716-9677-158ed4789ab5 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:53.961586 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-90841bac-df2f-4716-9677-158ed4789ab5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:53.977020 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90841bac-df2f-4716-9677-158ed4789ab5 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:53.985604 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-9a35aef5-cf32-4975-9d00-7dc708f296d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '220b20a472ef4bae838624c1994faf38', 'request_id': 'req-9a35aef5-cf32-4975-9d00-7dc708f296d4'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.002343 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-9a35aef5-cf32-4975-9d00-7dc708f296d4 admin admin] The token cache is being invalidate because project 220b20a472ef4bae838624c1994faf38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:54.002678 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-9a35aef5-cf32-4975-9d00-7dc708f296d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.003343 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-9a35aef5-cf32-4975-9d00-7dc708f296d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.005417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 915/1824] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:53 2022] DELETE /identity/v3/projects/220b20a472ef4bae838624c1994faf38 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:54.014288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35aef5-cf32-4975-9d00-7dc708f296d4 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:54.015715 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-9a35aef5-cf32-4975-9d00-7dc708f296d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:54.070667 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb7602-8564-468a-b645-26713c21c159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:54.071831 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb7602-8564-468a-b645-26713c21c159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:54.076355 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb7602-8564-468a-b645-26713c21c159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:54.076355 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb7602-8564-468a-b645-26713c21c159 admin admin] PATH_INFO: `/v3/projects/ad2e8f4a478644e79b8fe7bf4262e6e5` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:54.076355 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb7602-8564-468a-b645-26713c21c159 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad2e8f4a478644e79b8fe7bf4262e6e5)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:54.076355 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb7602-8564-468a-b645-26713c21c159 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:54.089193 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35aef5-cf32-4975-9d00-7dc708f296d4 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:54.093875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c668ee11-15fd-4f9d-9492-d053bc5aa126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:54.095885 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668ee11-15fd-4f9d-9492-d053bc5aa126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:54.096167 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668ee11-15fd-4f9d-9492-d053bc5aa126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:54.096456 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668ee11-15fd-4f9d-9492-d053bc5aa126 admin admin] PATH_INFO: `/v3/users/fb7ae41a0dab4776aa738fe480c55099` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:54.097723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c668ee11-15fd-4f9d-9492-d053bc5aa126 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb7ae41a0dab4776aa738fe480c55099)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:54.099834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c668ee11-15fd-4f9d-9492-d053bc5aa126 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:54.182587 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:54.192967 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-73eb7602-8564-468a-b645-26713c21c159 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad2e8f4a478644e79b8fe7bf4262e6e5', 'request_id': 'req-73eb7602-8564-468a-b645-26713c21c159'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.202994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-73eb7602-8564-468a-b645-26713c21c159 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad2e8f4a478644e79b8fe7bf4262e6e5', 'request_id': 'req-73eb7602-8564-468a-b645-26713c21c159'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.216971 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:54.327258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-73eb7602-8564-468a-b645-26713c21c159 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad2e8f4a478644e79b8fe7bf4262e6e5', 'request_id': 'req-73eb7602-8564-468a-b645-26713c21c159'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.339894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-73eb7602-8564-468a-b645-26713c21c159 admin admin] The token cache is being invalidate because project ad2e8f4a478644e79b8fe7bf4262e6e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:54.340261 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-73eb7602-8564-468a-b645-26713c21c159 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.340978 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-73eb7602-8564-468a-b645-26713c21c159 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.342679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 910/1825] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:53 2022] DELETE /identity/v3/projects/ad2e8f4a478644e79b8fe7bf4262e6e5 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:54.346863 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb7602-8564-468a-b645-26713c21c159 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:54.348261 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-73eb7602-8564-468a-b645-26713c21c159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:54.406191 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c668ee11-15fd-4f9d-9492-d053bc5aa126 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb7ae41a0dab4776aa738fe480c55099'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.413549 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c668ee11-15fd-4f9d-9492-d053bc5aa126 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb7ae41a0dab4776aa738fe480c55099'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.414268 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c668ee11-15fd-4f9d-9492-d053bc5aa126 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb7ae41a0dab4776aa738fe480c55099'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.414896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c668ee11-15fd-4f9d-9492-d053bc5aa126 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb7ae41a0dab4776aa738fe480c55099'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.431564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c668ee11-15fd-4f9d-9492-d053bc5aa126 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb7ae41a0dab4776aa738fe480c55099'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.440888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb7602-8564-468a-b645-26713c21c159 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:54.445988 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a7776-343c-4950-8255-6f9faa1257d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:54.447051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a7776-343c-4950-8255-6f9faa1257d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:54.447336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a7776-343c-4950-8255-6f9faa1257d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:54.447614 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a7776-343c-4950-8255-6f9faa1257d7 admin admin] PATH_INFO: `/v3/users/8c041acec9cc49adabb871298922116c` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:54.448875 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a7776-343c-4950-8255-6f9faa1257d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c041acec9cc49adabb871298922116c)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:54.450952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a7776-343c-4950-8255-6f9faa1257d7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:54.574155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 916/1826] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:54 2022] DELETE /identity/v3/users/fb7ae41a0dab4776aa738fe480c55099 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:54.772327 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-761a7776-343c-4950-8255-6f9faa1257d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c041acec9cc49adabb871298922116c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.779953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-761a7776-343c-4950-8255-6f9faa1257d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c041acec9cc49adabb871298922116c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.780676 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-761a7776-343c-4950-8255-6f9faa1257d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c041acec9cc49adabb871298922116c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.781235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-761a7776-343c-4950-8255-6f9faa1257d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c041acec9cc49adabb871298922116c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.793661 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-761a7776-343c-4950-8255-6f9faa1257d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c041acec9cc49adabb871298922116c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.894586 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 911/1827] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:54 2022] DELETE /identity/v3/users/8c041acec9cc49adabb871298922116c => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:55.120641 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c668ee11-15fd-4f9d-9492-d053bc5aa126 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:55.124950 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c668ee11-15fd-4f9d-9492-d053bc5aa126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:55.222443 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c668ee11-15fd-4f9d-9492-d053bc5aa126 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:55.295354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e3e62-3662-4be8-9087-0b24fe3affed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:55.296239 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e3e62-3662-4be8-9087-0b24fe3affed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:55.296600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e3e62-3662-4be8-9087-0b24fe3affed admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:55.296958 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e3e62-3662-4be8-9087-0b24fe3affed admin admin] PATH_INFO: `/v3/projects/3eb3e636461c4a319cc12584cc931158` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:55.298746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e3e62-3662-4be8-9087-0b24fe3affed admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eb3e636461c4a319cc12584cc931158)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:55.300731 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e3e62-3662-4be8-9087-0b24fe3affed admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:55.398320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a7776-343c-4950-8255-6f9faa1257d7 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:55.399828 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-761a7776-343c-4950-8255-6f9faa1257d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:55.441489 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a7776-343c-4950-8255-6f9faa1257d7 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:55.465489 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-696e3e62-3662-4be8-9087-0b24fe3affed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eb3e636461c4a319cc12584cc931158', 'request_id': 'req-696e3e62-3662-4be8-9087-0b24fe3affed'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:55.476316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-696e3e62-3662-4be8-9087-0b24fe3affed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eb3e636461c4a319cc12584cc931158', 'request_id': 'req-696e3e62-3662-4be8-9087-0b24fe3affed'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:55.479483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:55.521403 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:55.537073 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc520c-6b23-4d90-a348-0d74f80934d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:55.538054 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc520c-6b23-4d90-a348-0d74f80934d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:55.539124 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc520c-6b23-4d90-a348-0d74f80934d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:55.539124 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc520c-6b23-4d90-a348-0d74f80934d6 admin admin] PATH_INFO: `/v3/projects/bc47f1c70370473b9dc9e23ccb81534e` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:55.540245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fc520c-6b23-4d90-a348-0d74f80934d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc47f1c70370473b9dc9e23ccb81534e)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:55.542263 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fc520c-6b23-4d90-a348-0d74f80934d6 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:55.604143 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-696e3e62-3662-4be8-9087-0b24fe3affed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eb3e636461c4a319cc12584cc931158', 'request_id': 'req-696e3e62-3662-4be8-9087-0b24fe3affed'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:55.616087 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-696e3e62-3662-4be8-9087-0b24fe3affed admin admin] The token cache is being invalidate because project 3eb3e636461c4a319cc12584cc931158 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:55.616456 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-696e3e62-3662-4be8-9087-0b24fe3affed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:55.617179 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-696e3e62-3662-4be8-9087-0b24fe3affed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:55.619422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 917/1828] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:55 2022] DELETE /identity/v3/projects/3eb3e636461c4a319cc12584cc931158 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:55.623857 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89625a54-95e7-40f0-b781-c200e523dc3b None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:55.624151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89625a54-95e7-40f0-b781-c200e523dc3b None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:55.624437 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89625a54-95e7-40f0-b781-c200e523dc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:55.660135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-89625a54-95e7-40f0-b781-c200e523dc3b None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:55.660135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-89625a54-95e7-40f0-b781-c200e523dc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:55.662339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 918/1829] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:44:55.670195 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99056ca6-90a2-46ad-b38a-96403e97156b None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:55.670494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99056ca6-90a2-46ad-b38a-96403e97156b None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:55.670833 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99056ca6-90a2-46ad-b38a-96403e97156b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:55.692924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-99056ca6-90a2-46ad-b38a-96403e97156b None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:55.694816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-99056ca6-90a2-46ad-b38a-96403e97156b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:55.703284 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 919/1830] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:44:55.707698 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99056ca6-90a2-46ad-b38a-96403e97156b None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:55.718576 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99056ca6-90a2-46ad-b38a-96403e97156b None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:55.730226 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b9fc520c-6b23-4d90-a348-0d74f80934d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc47f1c70370473b9dc9e23ccb81534e', 'request_id': 'req-b9fc520c-6b23-4d90-a348-0d74f80934d6'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:55.737490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b9fc520c-6b23-4d90-a348-0d74f80934d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc47f1c70370473b9dc9e23ccb81534e', 'request_id': 'req-b9fc520c-6b23-4d90-a348-0d74f80934d6'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:55.784449 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea6744-1ec1-47ac-a36a-34a26cff59d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:55.785126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea6744-1ec1-47ac-a36a-34a26cff59d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:55.785518 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea6744-1ec1-47ac-a36a-34a26cff59d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:55.785729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea6744-1ec1-47ac-a36a-34a26cff59d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:55.786482 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea6744-1ec1-47ac-a36a-34a26cff59d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:55.786829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea6744-1ec1-47ac-a36a-34a26cff59d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:55.788459 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea6744-1ec1-47ac-a36a-34a26cff59d4 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:55.804550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 920/1831] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:44:55.805600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea6744-1ec1-47ac-a36a-34a26cff59d4 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:55.816713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea6744-1ec1-47ac-a36a-34a26cff59d4 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:55.821573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12a6b8-49ed-486a-b709-327c959299dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:55.822221 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12a6b8-49ed-486a-b709-327c959299dc admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:55.822503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12a6b8-49ed-486a-b709-327c959299dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:55.822807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12a6b8-49ed-486a-b709-327c959299dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:55.823503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12a6b8-49ed-486a-b709-327c959299dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:55.825610 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12a6b8-49ed-486a-b709-327c959299dc admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:55.837967 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b9fc520c-6b23-4d90-a348-0d74f80934d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc47f1c70370473b9dc9e23ccb81534e', 'request_id': 'req-b9fc520c-6b23-4d90-a348-0d74f80934d6'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:55.851369 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b9fc520c-6b23-4d90-a348-0d74f80934d6 admin admin] The token cache is being invalidate because project bc47f1c70370473b9dc9e23ccb81534e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:55.851793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b9fc520c-6b23-4d90-a348-0d74f80934d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:55.852540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b9fc520c-6b23-4d90-a348-0d74f80934d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:55.854366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 912/1832] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:55 2022] DELETE /identity/v3/projects/bc47f1c70370473b9dc9e23ccb81534e => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:55.858139 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc520c-6b23-4d90-a348-0d74f80934d6 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:55.859508 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b9fc520c-6b23-4d90-a348-0d74f80934d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:55.874171 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc520c-6b23-4d90-a348-0d74f80934d6 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:55.883137 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57344ac1-e908-4a8b-a856-ee2a26f07f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:55.884023 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57344ac1-e908-4a8b-a856-ee2a26f07f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:55.884301 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57344ac1-e908-4a8b-a856-ee2a26f07f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:55.884577 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57344ac1-e908-4a8b-a856-ee2a26f07f8a admin admin] PATH_INFO: `/v3/users/e7d84edcfbfb42b6a8e580a99da4feea` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:55.885863 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57344ac1-e908-4a8b-a856-ee2a26f07f8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7d84edcfbfb42b6a8e580a99da4feea)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:55.887895 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57344ac1-e908-4a8b-a856-ee2a26f07f8a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:55.966203 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 921/1833] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:55 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:55.970572 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12a6b8-49ed-486a-b709-327c959299dc admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:55.972336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-cb12a6b8-49ed-486a-b709-327c959299dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:56.035354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12a6b8-49ed-486a-b709-327c959299dc admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:56.039877 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74769835-5030-4676-b39c-07496074733c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:56.042806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74769835-5030-4676-b39c-07496074733c admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.042806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74769835-5030-4676-b39c-07496074733c admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.042806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74769835-5030-4676-b39c-07496074733c admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.042806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74769835-5030-4676-b39c-07496074733c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:56.043673 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74769835-5030-4676-b39c-07496074733c admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:56.086631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-57344ac1-e908-4a8b-a856-ee2a26f07f8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7d84edcfbfb42b6a8e580a99da4feea'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:56.093248 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-57344ac1-e908-4a8b-a856-ee2a26f07f8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7d84edcfbfb42b6a8e580a99da4feea'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:56.093917 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-57344ac1-e908-4a8b-a856-ee2a26f07f8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7d84edcfbfb42b6a8e580a99da4feea'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:56.095458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-57344ac1-e908-4a8b-a856-ee2a26f07f8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7d84edcfbfb42b6a8e580a99da4feea'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:56.106743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-57344ac1-e908-4a8b-a856-ee2a26f07f8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7d84edcfbfb42b6a8e580a99da4feea'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:56.146727 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 922/1834] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:55 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:56.150612 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74769835-5030-4676-b39c-07496074733c admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:56.152158 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-74769835-5030-4676-b39c-07496074733c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:56.163715 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74769835-5030-4676-b39c-07496074733c admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:56.168414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7230fb9-1158-45f5-984f-b40f1d4e3dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:56.169052 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7230fb9-1158-45f5-984f-b40f1d4e3dfb admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.169335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7230fb9-1158-45f5-984f-b40f1d4e3dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.169609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7230fb9-1158-45f5-984f-b40f1d4e3dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.170324 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7230fb9-1158-45f5-984f-b40f1d4e3dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:56.171944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7230fb9-1158-45f5-984f-b40f1d4e3dfb admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:56.181596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 913/1835] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:55 2022] DELETE /identity/v3/users/e7d84edcfbfb42b6a8e580a99da4feea => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:56.185782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 923/1836] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:44:56.186708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57344ac1-e908-4a8b-a856-ee2a26f07f8a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:56.242250 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:56.257186 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57344ac1-e908-4a8b-a856-ee2a26f07f8a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:56.261989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d877fc-29ae-4e35-8232-7fb82747ece4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:56.262716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d877fc-29ae-4e35-8232-7fb82747ece4 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.263008 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d877fc-29ae-4e35-8232-7fb82747ece4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.263284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d877fc-29ae-4e35-8232-7fb82747ece4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.264066 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d877fc-29ae-4e35-8232-7fb82747ece4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:56.265687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d877fc-29ae-4e35-8232-7fb82747ece4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:56.275864 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 914/1837] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:44:56.280090 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7230fb9-1158-45f5-984f-b40f1d4e3dfb admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:56.290541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7230fb9-1158-45f5-984f-b40f1d4e3dfb admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:56.295240 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e131860-e626-4a0e-9374-d9260d6a4b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:56.296115 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e131860-e626-4a0e-9374-d9260d6a4b2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.296515 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e131860-e626-4a0e-9374-d9260d6a4b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.296795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e131860-e626-4a0e-9374-d9260d6a4b2b admin admin] PATH_INFO: `/v3/projects/b326962d94be43a6898800a268dfd6f0/users/e9a76892d5ac45f384d5003d2dfd1b7d/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.312351 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e131860-e626-4a0e-9374-d9260d6a4b2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b326962d94be43a6898800a268dfd6f0, user_id=e9a76892d5ac45f384d5003d2dfd1b7d, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:56.315680 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e131860-e626-4a0e-9374-d9260d6a4b2b admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:56.318692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:INSERT=6,keystone:DELETE=3,keystone:SELECT=91 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:56.431702 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 924/1838] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:56 2022] PUT /identity/v3/projects/b326962d94be43a6898800a268dfd6f0/users/e9a76892d5ac45f384d5003d2dfd1b7d/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:56.442846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374f000-6a22-4c00-8ed6-41077c0003d6 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.443362 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374f000-6a22-4c00-8ed6-41077c0003d6 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.443463 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374f000-6a22-4c00-8ed6-41077c0003d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.479404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f374f000-6a22-4c00-8ed6-41077c0003d6 None None] MFA Rules not processed for user `e9a76892d5ac45f384d5003d2dfd1b7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:56.480805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f374f000-6a22-4c00-8ed6-41077c0003d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:56.575239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 915/1839] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:56.579843 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e131860-e626-4a0e-9374-d9260d6a4b2b admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:56.644547 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e131860-e626-4a0e-9374-d9260d6a4b2b admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:56.649173 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8f31b-b373-4331-9506-f13c73292879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:56.649839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8f31b-b373-4331-9506-f13c73292879 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.650122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8f31b-b373-4331-9506-f13c73292879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.650402 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8f31b-b373-4331-9506-f13c73292879 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.651149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8f31b-b373-4331-9506-f13c73292879 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:56.653093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8f31b-b373-4331-9506-f13c73292879 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:56.682214 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374f000-6a22-4c00-8ed6-41077c0003d6 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:56.687544 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f374f000-6a22-4c00-8ed6-41077c0003d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:56.702675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374f000-6a22-4c00-8ed6-41077c0003d6 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:56.710173 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e4d65-d526-46e0-b73b-522b1fc034e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:56.711134 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e4d65-d526-46e0-b73b-522b1fc034e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.711414 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e4d65-d526-46e0-b73b-522b1fc034e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.711917 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e4d65-d526-46e0-b73b-522b1fc034e9 admin admin] PATH_INFO: `/v3/projects/3b9503205d1b4ca7a113225c76ddda4b` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.713344 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e4d65-d526-46e0-b73b-522b1fc034e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b9503205d1b4ca7a113225c76ddda4b)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:56.724295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e4d65-d526-46e0-b73b-522b1fc034e9 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:56.791171 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 925/1840] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:56 2022] POST /identity/v3/projects => generated 389 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:56.795311 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8f31b-b373-4331-9506-f13c73292879 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:56.860500 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8f31b-b373-4331-9506-f13c73292879 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:56.866166 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af12f42-0191-493b-8d11-3473dd8abc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:56.867082 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af12f42-0191-493b-8d11-3473dd8abc03 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.867535 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af12f42-0191-493b-8d11-3473dd8abc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.867940 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af12f42-0191-493b-8d11-3473dd8abc03 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.868767 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af12f42-0191-493b-8d11-3473dd8abc03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:56.871007 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af12f42-0191-493b-8d11-3473dd8abc03 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:56.881421 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b06e4d65-d526-46e0-b73b-522b1fc034e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b9503205d1b4ca7a113225c76ddda4b', 'request_id': 'req-b06e4d65-d526-46e0-b73b-522b1fc034e9'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:56.888206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b06e4d65-d526-46e0-b73b-522b1fc034e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b9503205d1b4ca7a113225c76ddda4b', 'request_id': 'req-b06e4d65-d526-46e0-b73b-522b1fc034e9'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:56.986999 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b06e4d65-d526-46e0-b73b-522b1fc034e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b9503205d1b4ca7a113225c76ddda4b', 'request_id': 'req-b06e4d65-d526-46e0-b73b-522b1fc034e9'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:56.997072 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 926/1841] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:56 2022] POST /identity/v3/users => generated 301 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:56.997956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af12f42-0191-493b-8d11-3473dd8abc03 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:56.998985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b06e4d65-d526-46e0-b73b-522b1fc034e9 admin admin] The token cache is being invalidate because project 3b9503205d1b4ca7a113225c76ddda4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:56.999282 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b06e4d65-d526-46e0-b73b-522b1fc034e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:56.999990 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b06e4d65-d526-46e0-b73b-522b1fc034e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:57.007212 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 916/1842] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:56 2022] DELETE /identity/v3/projects/3b9503205d1b4ca7a113225c76ddda4b => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:57.009225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158a903-af44-4049-b5e2-5c9e9768bcdd None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.010183 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158a903-af44-4049-b5e2-5c9e9768bcdd None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.010183 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158a903-af44-4049-b5e2-5c9e9768bcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.015103 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af12f42-0191-493b-8d11-3473dd8abc03 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.039517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-2158a903-af44-4049-b5e2-5c9e9768bcdd None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:57.040783 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2158a903-af44-4049-b5e2-5c9e9768bcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:57.083899 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4edbf5-60e8-45e7-92e8-078f72e98ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.085059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4edbf5-60e8-45e7-92e8-078f72e98ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.085180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4edbf5-60e8-45e7-92e8-078f72e98ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.085709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4edbf5-60e8-45e7-92e8-078f72e98ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.086445 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4edbf5-60e8-45e7-92e8-078f72e98ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.088161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4edbf5-60e8-45e7-92e8-078f72e98ce5 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.106011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 927/1843] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:57.107214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4edbf5-60e8-45e7-92e8-078f72e98ce5 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.109116 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-2f4edbf5-60e8-45e7-92e8-078f72e98ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:57.114088 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:57.122258 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4edbf5-60e8-45e7-92e8-078f72e98ce5 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.128176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc8b27-00aa-4cff-ac22-eac2fb10e561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.129240 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc8b27-00aa-4cff-ac22-eac2fb10e561 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.129626 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc8b27-00aa-4cff-ac22-eac2fb10e561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.130003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc8b27-00aa-4cff-ac22-eac2fb10e561 admin admin] PATH_INFO: `/v3/projects/f8cd3e2113e84fafa08b9070467f8970/users/10fff3cf729744a08b61f7f064218e24/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.134710 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 917/1844] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:44:57.137057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75d29a-0838-4a25-a4e4-d137c66efe95 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.137600 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75d29a-0838-4a25-a4e4-d137c66efe95 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.137833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75d29a-0838-4a25-a4e4-d137c66efe95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.159039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc8b27-00aa-4cff-ac22-eac2fb10e561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8cd3e2113e84fafa08b9070467f8970, user_id=10fff3cf729744a08b61f7f064218e24, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.159039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc8b27-00aa-4cff-ac22-eac2fb10e561 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.159996 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-4c75d29a-0838-4a25-a4e4-d137c66efe95 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:57.161245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4c75d29a-0838-4a25-a4e4-d137c66efe95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:57.169890 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 918/1845] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:57.174481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75d29a-0838-4a25-a4e4-d137c66efe95 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.184200 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75d29a-0838-4a25-a4e4-d137c66efe95 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.189806 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017165cc-7981-4223-965b-143d69d47075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.190485 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017165cc-7981-4223-965b-143d69d47075 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.190806 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017165cc-7981-4223-965b-143d69d47075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.191083 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017165cc-7981-4223-965b-143d69d47075 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.191826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017165cc-7981-4223-965b-143d69d47075 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:57.192136 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017165cc-7981-4223-965b-143d69d47075 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.193766 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017165cc-7981-4223-965b-143d69d47075 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.206905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 919/1846] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:44:57.211754 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017165cc-7981-4223-965b-143d69d47075 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.222472 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017165cc-7981-4223-965b-143d69d47075 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.227559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b10f63-5d38-4324-9518-2c4a557826d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.228268 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b10f63-5d38-4324-9518-2c4a557826d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.228554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b10f63-5d38-4324-9518-2c4a557826d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.228835 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b10f63-5d38-4324-9518-2c4a557826d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.229541 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b10f63-5d38-4324-9518-2c4a557826d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.231540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b10f63-5d38-4324-9518-2c4a557826d8 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.243349 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 928/1847] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:57 2022] PUT /identity/v3/projects/f8cd3e2113e84fafa08b9070467f8970/users/10fff3cf729744a08b61f7f064218e24/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:57.254756 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc8b27-00aa-4cff-ac22-eac2fb10e561 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.283511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:57.316890 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc8b27-00aa-4cff-ac22-eac2fb10e561 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.320854 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0d89a-b2fa-4661-a125-2f1dbc8ffcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.321615 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0d89a-b2fa-4661-a125-2f1dbc8ffcca admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.321978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0d89a-b2fa-4661-a125-2f1dbc8ffcca admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.322333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0d89a-b2fa-4661-a125-2f1dbc8ffcca admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.323200 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0d89a-b2fa-4661-a125-2f1dbc8ffcca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.324882 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0d89a-b2fa-4661-a125-2f1dbc8ffcca admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.339604 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 929/1848] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:44:57.340958 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0d89a-b2fa-4661-a125-2f1dbc8ffcca admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.350825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0d89a-b2fa-4661-a125-2f1dbc8ffcca admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.355567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfec8b-9375-4acf-9386-a6b7c9696d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.356613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfec8b-9375-4acf-9386-a6b7c9696d58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.356903 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfec8b-9375-4acf-9386-a6b7c9696d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.357184 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfec8b-9375-4acf-9386-a6b7c9696d58 admin admin] PATH_INFO: `/v3/projects/f8cd3e2113e84fafa08b9070467f8970/users/10fff3cf729744a08b61f7f064218e24/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.360534 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 920/1849] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:57 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:57.360650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bfec8b-9375-4acf-9386-a6b7c9696d58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8cd3e2113e84fafa08b9070467f8970, user_id=10fff3cf729744a08b61f7f064218e24, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.361620 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b10f63-5d38-4324-9518-2c4a557826d8 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.362339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bfec8b-9375-4acf-9386-a6b7c9696d58 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.373117 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 930/1850] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:57 2022] PUT /identity/v3/projects/f8cd3e2113e84fafa08b9070467f8970/users/10fff3cf729744a08b61f7f064218e24/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:57.383738 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bb74e-f36e-4114-b607-73f69373850b None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.384036 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bb74e-f36e-4114-b607-73f69373850b None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.384322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bb74e-f36e-4114-b607-73f69373850b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.410515 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-ea3bb74e-f36e-4114-b607-73f69373850b None None] MFA Rules not processed for user `10fff3cf729744a08b61f7f064218e24`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:57.411654 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-ea3bb74e-f36e-4114-b607-73f69373850b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:57.421762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b10f63-5d38-4324-9518-2c4a557826d8 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.478715 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a0de0-2345-4891-ba3c-7f7a3534471a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.479364 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 931/1851] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:44:57.479474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a0de0-2345-4891-ba3c-7f7a3534471a admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.479670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a0de0-2345-4891-ba3c-7f7a3534471a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.479950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a0de0-2345-4891-ba3c-7f7a3534471a admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.480715 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a0de0-2345-4891-ba3c-7f7a3534471a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.482677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a0de0-2345-4891-ba3c-7f7a3534471a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.495963 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bb74e-f36e-4114-b607-73f69373850b None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.497483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-ea3bb74e-f36e-4114-b607-73f69373850b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:57.554565 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bb74e-f36e-4114-b607-73f69373850b None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.558420 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d1a9b-ef3b-4b75-821c-f221df8ba55c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.559138 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d1a9b-ef3b-4b75-821c-f221df8ba55c service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.559426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d1a9b-ef3b-4b75-821c-f221df8ba55c service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.559704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d1a9b-ef3b-4b75-821c-f221df8ba55c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.569141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d1a9b-ef3b-4b75-821c-f221df8ba55c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.570630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d1a9b-ef3b-4b75-821c-f221df8ba55c service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.580884 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 932/1852] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:44:57.602742 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 921/1853] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:57 2022] POST /identity/v3/users => generated 289 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:57.604331 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d1a9b-ef3b-4b75-821c-f221df8ba55c service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.614903 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d1a9b-ef3b-4b75-821c-f221df8ba55c service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.625345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdacab-1a30-49af-b8e4-2cc7be7d38a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.625345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdacab-1a30-49af-b8e4-2cc7be7d38a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.625345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdacab-1a30-49af-b8e4-2cc7be7d38a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.625345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdacab-1a30-49af-b8e4-2cc7be7d38a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.626899 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bdacab-1a30-49af-b8e4-2cc7be7d38a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.628880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bdacab-1a30-49af-b8e4-2cc7be7d38a6 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.643468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 933/1854] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:57.644995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a0de0-2345-4891-ba3c-7f7a3534471a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.655252 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a0de0-2345-4891-ba3c-7f7a3534471a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.661017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689d6b8-8d39-40af-8f2d-cec7256855a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.661629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689d6b8-8d39-40af-8f2d-cec7256855a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.661915 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689d6b8-8d39-40af-8f2d-cec7256855a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.662194 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689d6b8-8d39-40af-8f2d-cec7256855a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.662972 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689d6b8-8d39-40af-8f2d-cec7256855a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.664650 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689d6b8-8d39-40af-8f2d-cec7256855a7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.676580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 922/1855] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:44:57.680580 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdacab-1a30-49af-b8e4-2cc7be7d38a6 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.693495 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdacab-1a30-49af-b8e4-2cc7be7d38a6 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.698886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594988e3-c0f8-49df-abad-d4a2c9f9b556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.700212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594988e3-c0f8-49df-abad-d4a2c9f9b556 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.700566 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594988e3-c0f8-49df-abad-d4a2c9f9b556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.700907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594988e3-c0f8-49df-abad-d4a2c9f9b556 admin admin] PATH_INFO: `/v3/projects/abb8c04ae0ec4a948826540a2d5e947d/users/bea62fd563dc424cb9beee5a5b1acb52/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.717837 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689d6b8-8d39-40af-8f2d-cec7256855a7 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.722167 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594988e3-c0f8-49df-abad-d4a2c9f9b556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb8c04ae0ec4a948826540a2d5e947d, user_id=bea62fd563dc424cb9beee5a5b1acb52, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.724972 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594988e3-c0f8-49df-abad-d4a2c9f9b556 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.729364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689d6b8-8d39-40af-8f2d-cec7256855a7 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.734171 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabac8ba-9dc3-4c75-bbcb-2c11a20f8cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.735198 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabac8ba-9dc3-4c75-bbcb-2c11a20f8cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.735553 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabac8ba-9dc3-4c75-bbcb-2c11a20f8cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.736106 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabac8ba-9dc3-4c75-bbcb-2c11a20f8cf2 admin admin] PATH_INFO: `/v3/users/e9a76892d5ac45f384d5003d2dfd1b7d` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.737470 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabac8ba-9dc3-4c75-bbcb-2c11a20f8cf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9a76892d5ac45f384d5003d2dfd1b7d)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.739546 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabac8ba-9dc3-4c75-bbcb-2c11a20f8cf2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.751212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:57.807998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 934/1856] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:57 2022] PUT /identity/v3/projects/abb8c04ae0ec4a948826540a2d5e947d/users/bea62fd563dc424cb9beee5a5b1acb52/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:57.820117 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc721cc-f52e-44c4-a3cb-dc9216270b0a None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.820399 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc721cc-f52e-44c4-a3cb-dc9216270b0a None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.820672 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc721cc-f52e-44c4-a3cb-dc9216270b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.850928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-7bc721cc-f52e-44c4-a3cb-dc9216270b0a None None] MFA Rules not processed for user `bea62fd563dc424cb9beee5a5b1acb52`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:57.852167 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-7bc721cc-f52e-44c4-a3cb-dc9216270b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:57.918472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 935/1857] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:44:57.931715 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc721cc-f52e-44c4-a3cb-dc9216270b0a None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.933504 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-7bc721cc-f52e-44c4-a3cb-dc9216270b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:57.952054 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cabac8ba-9dc3-4c75-bbcb-2c11a20f8cf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9a76892d5ac45f384d5003d2dfd1b7d'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:57.961141 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cabac8ba-9dc3-4c75-bbcb-2c11a20f8cf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9a76892d5ac45f384d5003d2dfd1b7d'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:57.961792 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cabac8ba-9dc3-4c75-bbcb-2c11a20f8cf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9a76892d5ac45f384d5003d2dfd1b7d'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:57.962477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cabac8ba-9dc3-4c75-bbcb-2c11a20f8cf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9a76892d5ac45f384d5003d2dfd1b7d'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:57.969466 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:57.978747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cabac8ba-9dc3-4c75-bbcb-2c11a20f8cf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9a76892d5ac45f384d5003d2dfd1b7d'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:58.007798 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc721cc-f52e-44c4-a3cb-dc9216270b0a None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:58.011928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b9858-a520-482c-a500-8362c6a9c484 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6105838cbf1409587ee7d9e3be554f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:58.012659 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b9858-a520-482c-a500-8362c6a9c484 service glance] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:58.013016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b9858-a520-482c-a500-8362c6a9c484 service glance] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:58.013362 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b9858-a520-482c-a500-8362c6a9c484 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:58.015327 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-8d7b9858-a520-482c-a500-8362c6a9c484 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:58.028170 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b9858-a520-482c-a500-8362c6a9c484 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:58.029621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b9858-a520-482c-a500-8362c6a9c484 service glance] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:58.039492 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 936/1858] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:44:58.076746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 923/1859] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:57 2022] DELETE /identity/v3/users/e9a76892d5ac45f384d5003d2dfd1b7d => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:58.078550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c3df7-bfe7-43f3-b817-4a69419c6b88 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:58.080459 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c3df7-bfe7-43f3-b817-4a69419c6b88 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:58.080459 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c3df7-bfe7-43f3-b817-4a69419c6b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:58.110238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-a38c3df7-bfe7-43f3-b817-4a69419c6b88 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:58.113039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a38c3df7-bfe7-43f3-b817-4a69419c6b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:58.207108 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 937/1860] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:44:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:58.216859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabac8ba-9dc3-4c75-bbcb-2c11a20f8cf2 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:58.218253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-cabac8ba-9dc3-4c75-bbcb-2c11a20f8cf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:58.279638 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabac8ba-9dc3-4c75-bbcb-2c11a20f8cf2 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:58.283506 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228b140-0830-4555-9bf7-0dea58153337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:58.284160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228b140-0830-4555-9bf7-0dea58153337 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:58.284401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:58.284518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228b140-0830-4555-9bf7-0dea58153337 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:58.284817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228b140-0830-4555-9bf7-0dea58153337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:58.294877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228b140-0830-4555-9bf7-0dea58153337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:58.296326 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228b140-0830-4555-9bf7-0dea58153337 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:58.306579 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 924/1861] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:58.861503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c3df7-bfe7-43f3-b817-4a69419c6b88 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:58.862971 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a38c3df7-bfe7-43f3-b817-4a69419c6b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:58.877112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c3df7-bfe7-43f3-b817-4a69419c6b88 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:58.881620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7bb8f-2ccb-4561-8981-5f3b1b183ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:58.883113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7bb8f-2ccb-4561-8981-5f3b1b183ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:58.883578 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7bb8f-2ccb-4561-8981-5f3b1b183ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:58.883856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7bb8f-2ccb-4561-8981-5f3b1b183ef8 admin admin] PATH_INFO: `/v3/projects/b326962d94be43a6898800a268dfd6f0` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:58.885041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7bb8f-2ccb-4561-8981-5f3b1b183ef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b326962d94be43a6898800a268dfd6f0)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:58.887013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7bb8f-2ccb-4561-8981-5f3b1b183ef8 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:59.029044 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6ad7bb8f-2ccb-4561-8981-5f3b1b183ef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b326962d94be43a6898800a268dfd6f0', 'request_id': 'req-6ad7bb8f-2ccb-4561-8981-5f3b1b183ef8'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:59.038467 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6ad7bb8f-2ccb-4561-8981-5f3b1b183ef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b326962d94be43a6898800a268dfd6f0', 'request_id': 'req-6ad7bb8f-2ccb-4561-8981-5f3b1b183ef8'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:59.147125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6ad7bb8f-2ccb-4561-8981-5f3b1b183ef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b326962d94be43a6898800a268dfd6f0', 'request_id': 'req-6ad7bb8f-2ccb-4561-8981-5f3b1b183ef8'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:59.160159 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6ad7bb8f-2ccb-4561-8981-5f3b1b183ef8 admin admin] The token cache is being invalidate because project b326962d94be43a6898800a268dfd6f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:59.160465 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6ad7bb8f-2ccb-4561-8981-5f3b1b183ef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:59.165600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6ad7bb8f-2ccb-4561-8981-5f3b1b183ef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:59.165600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 938/1862] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:58 2022] DELETE /identity/v3/projects/b326962d94be43a6898800a268dfd6f0 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:59.179859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228b140-0830-4555-9bf7-0dea58153337 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:59.193223 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228b140-0830-4555-9bf7-0dea58153337 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:59.285577 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd480c-7823-4b9d-82de-30f8cdf03076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:59.286482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd480c-7823-4b9d-82de-30f8cdf03076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:59.286852 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd480c-7823-4b9d-82de-30f8cdf03076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:59.287131 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd480c-7823-4b9d-82de-30f8cdf03076 admin admin] PATH_INFO: `/v3/users/10fff3cf729744a08b61f7f064218e24` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:59.291892 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd480c-7823-4b9d-82de-30f8cdf03076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10fff3cf729744a08b61f7f064218e24)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:59.291892 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd480c-7823-4b9d-82de-30f8cdf03076 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:59.538474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-dafd480c-7823-4b9d-82de-30f8cdf03076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10fff3cf729744a08b61f7f064218e24'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:59.545583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-dafd480c-7823-4b9d-82de-30f8cdf03076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10fff3cf729744a08b61f7f064218e24'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:59.546291 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-dafd480c-7823-4b9d-82de-30f8cdf03076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10fff3cf729744a08b61f7f064218e24'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:59.546922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-dafd480c-7823-4b9d-82de-30f8cdf03076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10fff3cf729744a08b61f7f064218e24'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:59.548188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:59.560266 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-dafd480c-7823-4b9d-82de-30f8cdf03076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10fff3cf729744a08b61f7f064218e24'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:59.691365 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 925/1863] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:59 2022] DELETE /identity/v3/users/10fff3cf729744a08b61f7f064218e24 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:00.197664 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7bb8f-2ccb-4561-8981-5f3b1b183ef8 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:00.199586 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-6ad7bb8f-2ccb-4561-8981-5f3b1b183ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:00.218008 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7bb8f-2ccb-4561-8981-5f3b1b183ef8 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:00.286569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac65c8-3e68-4dc9-bd68-be474ddd4364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:00.287402 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac65c8-3e68-4dc9-bd68-be474ddd4364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:00.287689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac65c8-3e68-4dc9-bd68-be474ddd4364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:00.287970 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac65c8-3e68-4dc9-bd68-be474ddd4364 admin admin] PATH_INFO: `/v3/projects/f8cd3e2113e84fafa08b9070467f8970` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:00.289179 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac65c8-3e68-4dc9-bd68-be474ddd4364 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8cd3e2113e84fafa08b9070467f8970)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:00.291154 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac65c8-3e68-4dc9-bd68-be474ddd4364 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:00.427708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-cdac65c8-3e68-4dc9-bd68-be474ddd4364 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8cd3e2113e84fafa08b9070467f8970', 'request_id': 'req-cdac65c8-3e68-4dc9-bd68-be474ddd4364'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:00.434393 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-cdac65c8-3e68-4dc9-bd68-be474ddd4364 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8cd3e2113e84fafa08b9070467f8970', 'request_id': 'req-cdac65c8-3e68-4dc9-bd68-be474ddd4364'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:00.511884 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-cdac65c8-3e68-4dc9-bd68-be474ddd4364 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8cd3e2113e84fafa08b9070467f8970', 'request_id': 'req-cdac65c8-3e68-4dc9-bd68-be474ddd4364'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:00.524768 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-cdac65c8-3e68-4dc9-bd68-be474ddd4364 admin admin] The token cache is being invalidate because project f8cd3e2113e84fafa08b9070467f8970 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:00.525047 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-cdac65c8-3e68-4dc9-bd68-be474ddd4364 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:00.525709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-cdac65c8-3e68-4dc9-bd68-be474ddd4364 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:00.531349 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 939/1864] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:00 2022] DELETE /identity/v3/projects/f8cd3e2113e84fafa08b9070467f8970 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:00.533487 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf38ca-073b-49ef-a800-6cfaf5a62ea3 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:00.533958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf38ca-073b-49ef-a800-6cfaf5a62ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:00.534056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf38ca-073b-49ef-a800-6cfaf5a62ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:00.579402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-eacf38ca-073b-49ef-a800-6cfaf5a62ea3 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:00.582201 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-eacf38ca-073b-49ef-a800-6cfaf5a62ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:00.671855 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 926/1865] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:00.679871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e9d91-39a8-4879-902a-2127510641fa None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:00.680259 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e9d91-39a8-4879-902a-2127510641fa None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:00.680495 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e9d91-39a8-4879-902a-2127510641fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:00.690467 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:00.703320 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-292e9d91-39a8-4879-902a-2127510641fa None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:00.706492 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-292e9d91-39a8-4879-902a-2127510641fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:00.715656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 940/1866] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:00.719733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf38ca-073b-49ef-a800-6cfaf5a62ea3 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:00.729928 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf38ca-073b-49ef-a800-6cfaf5a62ea3 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:00.735319 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9149a0-a438-4d6a-a9d6-eee6b51ac62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:00.736367 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9149a0-a438-4d6a-a9d6-eee6b51ac62e admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:00.736649 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9149a0-a438-4d6a-a9d6-eee6b51ac62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:00.736921 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9149a0-a438-4d6a-a9d6-eee6b51ac62e admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:00.737651 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9149a0-a438-4d6a-a9d6-eee6b51ac62e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:00.737956 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9149a0-a438-4d6a-a9d6-eee6b51ac62e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:00.739772 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9149a0-a438-4d6a-a9d6-eee6b51ac62e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:00.753949 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 927/1867] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:00.759298 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e9d91-39a8-4879-902a-2127510641fa None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:00.771178 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e9d91-39a8-4879-902a-2127510641fa None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:00.778293 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce950de-6c1a-4f01-8022-4a46ff045b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:00.779561 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce950de-6c1a-4f01-8022-4a46ff045b6c admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:00.779561 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce950de-6c1a-4f01-8022-4a46ff045b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:00.779561 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce950de-6c1a-4f01-8022-4a46ff045b6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:00.780386 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce950de-6c1a-4f01-8022-4a46ff045b6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:00.782866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce950de-6c1a-4f01-8022-4a46ff045b6c admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:00.953209 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 941/1868] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:00 2022] POST /identity/v3/projects => generated 371 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:00.955328 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9149a0-a438-4d6a-a9d6-eee6b51ac62e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:00.967624 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9149a0-a438-4d6a-a9d6-eee6b51ac62e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:00.975727 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce950de-6c1a-4f01-8022-4a46ff045b6c admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:00.978248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-fce950de-6c1a-4f01-8022-4a46ff045b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:01.036458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33209b-584e-448f-a67a-14b461664d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:01.037444 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33209b-584e-448f-a67a-14b461664d1b admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:01.037728 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33209b-584e-448f-a67a-14b461664d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:01.038005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33209b-584e-448f-a67a-14b461664d1b admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:01.038734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33209b-584e-448f-a67a-14b461664d1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:01.040792 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33209b-584e-448f-a67a-14b461664d1b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:01.053529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce950de-6c1a-4f01-8022-4a46ff045b6c admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:01.060014 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad29d9-e819-4a27-b058-d345b658e9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:01.060931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad29d9-e819-4a27-b058-d345b658e9d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:01.061220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad29d9-e819-4a27-b058-d345b658e9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:01.061502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad29d9-e819-4a27-b058-d345b658e9d0 admin admin] PATH_INFO: `/v3/users/12840922e0a0477a92ff9be10d89e7c5` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:01.062802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad29d9-e819-4a27-b058-d345b658e9d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12840922e0a0477a92ff9be10d89e7c5)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:01.065433 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad29d9-e819-4a27-b058-d345b658e9d0 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:01.192633 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 928/1869] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:00 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:01.197614 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33209b-584e-448f-a67a-14b461664d1b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:01.207514 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33209b-584e-448f-a67a-14b461664d1b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:01.212588 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514ad23-fe88-4a92-bd59-1cbc659e25da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:01.213308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514ad23-fe88-4a92-bd59-1cbc659e25da admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:01.214010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514ad23-fe88-4a92-bd59-1cbc659e25da admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:01.214242 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514ad23-fe88-4a92-bd59-1cbc659e25da admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:01.215032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514ad23-fe88-4a92-bd59-1cbc659e25da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:01.216639 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514ad23-fe88-4a92-bd59-1cbc659e25da admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:01.229337 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 929/1870] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:45:01.235781 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514ad23-fe88-4a92-bd59-1cbc659e25da admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:01.246279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514ad23-fe88-4a92-bd59-1cbc659e25da admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:01.251693 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b881d1b-7b64-42e5-a5a7-111261a98f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:01.252070 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b881d1b-7b64-42e5-a5a7-111261a98f5c admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:01.252429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b881d1b-7b64-42e5-a5a7-111261a98f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:01.252779 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b881d1b-7b64-42e5-a5a7-111261a98f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:01.253737 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b881d1b-7b64-42e5-a5a7-111261a98f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:01.255506 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b881d1b-7b64-42e5-a5a7-111261a98f5c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:01.268257 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 930/1871] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:01.274839 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b881d1b-7b64-42e5-a5a7-111261a98f5c admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:01.287662 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b881d1b-7b64-42e5-a5a7-111261a98f5c admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:01.292561 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd97c89-bdae-43c8-9481-082f4a40a38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:01.293431 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd97c89-bdae-43c8-9481-082f4a40a38c admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:01.293711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd97c89-bdae-43c8-9481-082f4a40a38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:01.293987 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd97c89-bdae-43c8-9481-082f4a40a38c admin admin] PATH_INFO: `/v3/projects/8d329f6f8f524ec1af47bc6fb3c65da7/users/07057d86c86a4e56bb35330b4ffa49f6/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:01.313138 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd97c89-bdae-43c8-9481-082f4a40a38c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d329f6f8f524ec1af47bc6fb3c65da7, user_id=07057d86c86a4e56bb35330b4ffa49f6, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:01.315922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd97c89-bdae-43c8-9481-082f4a40a38c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:01.369088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-27ad29d9-e819-4a27-b058-d345b658e9d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12840922e0a0477a92ff9be10d89e7c5'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:01.378040 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-27ad29d9-e819-4a27-b058-d345b658e9d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12840922e0a0477a92ff9be10d89e7c5'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:01.378838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-27ad29d9-e819-4a27-b058-d345b658e9d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12840922e0a0477a92ff9be10d89e7c5'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:01.379615 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-27ad29d9-e819-4a27-b058-d345b658e9d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12840922e0a0477a92ff9be10d89e7c5'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:01.394569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-27ad29d9-e819-4a27-b058-d345b658e9d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12840922e0a0477a92ff9be10d89e7c5'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:01.398751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 931/1872] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:01 2022] PUT /identity/v3/projects/8d329f6f8f524ec1af47bc6fb3c65da7/users/07057d86c86a4e56bb35330b4ffa49f6/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:45:01.410463 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588989d-0daa-42cd-a5a1-089636100ed6 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:01.410797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588989d-0daa-42cd-a5a1-089636100ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:01.411077 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588989d-0daa-42cd-a5a1-089636100ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:01.417528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:01.436731 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-4588989d-0daa-42cd-a5a1-089636100ed6 None None] MFA Rules not processed for user `07057d86c86a4e56bb35330b4ffa49f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:01.437969 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4588989d-0daa-42cd-a5a1-089636100ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:01.501352 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 942/1873] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:00 2022] DELETE /identity/v3/users/12840922e0a0477a92ff9be10d89e7c5 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:01.505692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5ecfc-5cc2-4232-8115-252433be718d None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:01.506291 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5ecfc-5cc2-4232-8115-252433be718d None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:01.506624 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5ecfc-5cc2-4232-8115-252433be718d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:01.507657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 932/1874] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:45:01.519438 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588989d-0daa-42cd-a5a1-089636100ed6 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:01.521188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4588989d-0daa-42cd-a5a1-089636100ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:01.526820 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-d4a5ecfc-5cc2-4232-8115-252433be718d None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:01.528047 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d4a5ecfc-5cc2-4232-8115-252433be718d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:01.579936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588989d-0daa-42cd-a5a1-089636100ed6 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:01.589813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 943/1875] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:45:01.600669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5ecfc-5cc2-4232-8115-252433be718d None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:01.604078 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d4a5ecfc-5cc2-4232-8115-252433be718d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:01.642851 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710fc05-504e-4c8e-98a0-9348c8c43757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:01.643627 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710fc05-504e-4c8e-98a0-9348c8c43757 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:01.643994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710fc05-504e-4c8e-98a0-9348c8c43757 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:01.644492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710fc05-504e-4c8e-98a0-9348c8c43757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:01.677488 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5ecfc-5cc2-4232-8115-252433be718d None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:01.683507 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5704955-ed2e-4b6d-84bd-b364ba935c29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:01.684767 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5704955-ed2e-4b6d-84bd-b364ba935c29 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:01.685221 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5704955-ed2e-4b6d-84bd-b364ba935c29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:01.685517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5704955-ed2e-4b6d-84bd-b364ba935c29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:01.698898 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5704955-ed2e-4b6d-84bd-b364ba935c29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:01.700561 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5704955-ed2e-4b6d-84bd-b364ba935c29 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:01.712522 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 944/1876] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:45:01.715042 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710fc05-504e-4c8e-98a0-9348c8c43757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:01.725780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710fc05-504e-4c8e-98a0-9348c8c43757 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:01.726700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 933/1877] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:45:01.746671 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5704955-ed2e-4b6d-84bd-b364ba935c29 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:01.748567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a5704955-ed2e-4b6d-84bd-b364ba935c29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:01.762321 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5704955-ed2e-4b6d-84bd-b364ba935c29 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:01.767918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca3e20-835b-47c3-bf9c-9558b94d1eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:01.768864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca3e20-835b-47c3-bf9c-9558b94d1eee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:01.769955 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca3e20-835b-47c3-bf9c-9558b94d1eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:01.770387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca3e20-835b-47c3-bf9c-9558b94d1eee admin admin] PATH_INFO: `/v3/users/07057d86c86a4e56bb35330b4ffa49f6` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:01.771928 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca3e20-835b-47c3-bf9c-9558b94d1eee admin admin] RBAC: Authorizing `identity:delete_user(user_id=07057d86c86a4e56bb35330b4ffa49f6)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:01.774083 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca3e20-835b-47c3-bf9c-9558b94d1eee admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:02.010087 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3fca3e20-835b-47c3-bf9c-9558b94d1eee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07057d86c86a4e56bb35330b4ffa49f6'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:02.017176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3fca3e20-835b-47c3-bf9c-9558b94d1eee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07057d86c86a4e56bb35330b4ffa49f6'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:02.017864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3fca3e20-835b-47c3-bf9c-9558b94d1eee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07057d86c86a4e56bb35330b4ffa49f6'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:02.018476 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3fca3e20-835b-47c3-bf9c-9558b94d1eee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07057d86c86a4e56bb35330b4ffa49f6'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:02.030186 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3fca3e20-835b-47c3-bf9c-9558b94d1eee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07057d86c86a4e56bb35330b4ffa49f6'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:02.065729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:02.128397 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 945/1878] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:01 2022] DELETE /identity/v3/users/07057d86c86a4e56bb35330b4ffa49f6 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:02.137745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b6aea-5dd0-4941-a628-edfb4449bd18 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:02.138210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b6aea-5dd0-4941-a628-edfb4449bd18 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:02.138772 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b6aea-5dd0-4941-a628-edfb4449bd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:02.179750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c83b6aea-5dd0-4941-a628-edfb4449bd18 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:02.181255 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c83b6aea-5dd0-4941-a628-edfb4449bd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:02.293330 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 934/1879] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:45:02.308281 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca3e20-835b-47c3-bf9c-9558b94d1eee admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:02.309717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-3fca3e20-835b-47c3-bf9c-9558b94d1eee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:02.372601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca3e20-835b-47c3-bf9c-9558b94d1eee admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:02.377873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2a3e9-cb5f-4ce0-89b4-b04b60a1aa07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:02.378614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2a3e9-cb5f-4ce0-89b4-b04b60a1aa07 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:02.378910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2a3e9-cb5f-4ce0-89b4-b04b60a1aa07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:02.379212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2a3e9-cb5f-4ce0-89b4-b04b60a1aa07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:02.388605 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2a3e9-cb5f-4ce0-89b4-b04b60a1aa07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:02.390050 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2a3e9-cb5f-4ce0-89b4-b04b60a1aa07 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:02.399905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 946/1880] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:02.421766 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:02.497866 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b6aea-5dd0-4941-a628-edfb4449bd18 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:02.498254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c83b6aea-5dd0-4941-a628-edfb4449bd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:02.517702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b6aea-5dd0-4941-a628-edfb4449bd18 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:02.523291 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ab77e-8e9c-432c-965c-5720b562c5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:02.524555 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ab77e-8e9c-432c-965c-5720b562c5df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:02.524675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ab77e-8e9c-432c-965c-5720b562c5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:02.525034 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ab77e-8e9c-432c-965c-5720b562c5df admin admin] PATH_INFO: `/v3/projects/68b3200de93347789acada3e464271d7` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:02.526361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ab77e-8e9c-432c-965c-5720b562c5df admin admin] RBAC: Authorizing `identity:delete_project(project_id=68b3200de93347789acada3e464271d7)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:02.528467 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ab77e-8e9c-432c-965c-5720b562c5df admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:02.680835 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4c9ab77e-8e9c-432c-965c-5720b562c5df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68b3200de93347789acada3e464271d7', 'request_id': 'req-4c9ab77e-8e9c-432c-965c-5720b562c5df'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:02.687448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4c9ab77e-8e9c-432c-965c-5720b562c5df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68b3200de93347789acada3e464271d7', 'request_id': 'req-4c9ab77e-8e9c-432c-965c-5720b562c5df'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:02.791971 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4c9ab77e-8e9c-432c-965c-5720b562c5df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68b3200de93347789acada3e464271d7', 'request_id': 'req-4c9ab77e-8e9c-432c-965c-5720b562c5df'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:02.803523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4c9ab77e-8e9c-432c-965c-5720b562c5df admin admin] The token cache is being invalidate because project 68b3200de93347789acada3e464271d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:02.803817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4c9ab77e-8e9c-432c-965c-5720b562c5df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:02.804458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4c9ab77e-8e9c-432c-965c-5720b562c5df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:02.806051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 935/1881] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:02 2022] DELETE /identity/v3/projects/68b3200de93347789acada3e464271d7 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:02.810102 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2a3e9-cb5f-4ce0-89b4-b04b60a1aa07 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:02.820578 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2a3e9-cb5f-4ce0-89b4-b04b60a1aa07 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:02.883053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e6c54-78f6-4e4d-a819-9a4ee7cc9481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:02.883935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e6c54-78f6-4e4d-a819-9a4ee7cc9481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:02.884205 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e6c54-78f6-4e4d-a819-9a4ee7cc9481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:02.884483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e6c54-78f6-4e4d-a819-9a4ee7cc9481 admin admin] PATH_INFO: `/v3/users/19a07e583d404b658435a58898780279` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:02.885676 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e6c54-78f6-4e4d-a819-9a4ee7cc9481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19a07e583d404b658435a58898780279)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:02.887632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e6c54-78f6-4e4d-a819-9a4ee7cc9481 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:02.897052 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ab77e-8e9c-432c-965c-5720b562c5df admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:02.899755 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4c9ab77e-8e9c-432c-965c-5720b562c5df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:02.927274 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ab77e-8e9c-432c-965c-5720b562c5df admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:02.933459 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d2c2d7-c5e3-4e98-b1f4-506f59f889f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:02.934763 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2c2d7-c5e3-4e98-b1f4-506f59f889f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:02.935140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2c2d7-c5e3-4e98-b1f4-506f59f889f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:02.935508 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2c2d7-c5e3-4e98-b1f4-506f59f889f0 admin admin] PATH_INFO: `/v3/projects/8d329f6f8f524ec1af47bc6fb3c65da7` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:02.937092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d2c2d7-c5e3-4e98-b1f4-506f59f889f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d329f6f8f524ec1af47bc6fb3c65da7)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:02.939382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d2c2d7-c5e3-4e98-b1f4-506f59f889f0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:02.986711 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:03.105871 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-25d2c2d7-c5e3-4e98-b1f4-506f59f889f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d329f6f8f524ec1af47bc6fb3c65da7', 'request_id': 'req-25d2c2d7-c5e3-4e98-b1f4-506f59f889f0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:03.112190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-077e6c54-78f6-4e4d-a819-9a4ee7cc9481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19a07e583d404b658435a58898780279'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:03.113701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-25d2c2d7-c5e3-4e98-b1f4-506f59f889f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d329f6f8f524ec1af47bc6fb3c65da7', 'request_id': 'req-25d2c2d7-c5e3-4e98-b1f4-506f59f889f0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:03.120062 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-077e6c54-78f6-4e4d-a819-9a4ee7cc9481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19a07e583d404b658435a58898780279'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:03.121017 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-077e6c54-78f6-4e4d-a819-9a4ee7cc9481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19a07e583d404b658435a58898780279'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:03.121628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-077e6c54-78f6-4e4d-a819-9a4ee7cc9481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19a07e583d404b658435a58898780279'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:03.133829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-077e6c54-78f6-4e4d-a819-9a4ee7cc9481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19a07e583d404b658435a58898780279'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:03.197273 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-25d2c2d7-c5e3-4e98-b1f4-506f59f889f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d329f6f8f524ec1af47bc6fb3c65da7', 'request_id': 'req-25d2c2d7-c5e3-4e98-b1f4-506f59f889f0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:03.209682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-25d2c2d7-c5e3-4e98-b1f4-506f59f889f0 admin admin] The token cache is being invalidate because project 8d329f6f8f524ec1af47bc6fb3c65da7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:03.209966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-25d2c2d7-c5e3-4e98-b1f4-506f59f889f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:03.210648 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-25d2c2d7-c5e3-4e98-b1f4-506f59f889f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:03.212513 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 936/1882] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:02 2022] DELETE /identity/v3/projects/8d329f6f8f524ec1af47bc6fb3c65da7 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:03.216797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee750f17-4dd0-4fc4-b8a6-653e40b573d9 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:03.217079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee750f17-4dd0-4fc4-b8a6-653e40b573d9 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:03.217352 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee750f17-4dd0-4fc4-b8a6-653e40b573d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:03.235026 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ee750f17-4dd0-4fc4-b8a6-653e40b573d9 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:03.236228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ee750f17-4dd0-4fc4-b8a6-653e40b573d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:03.241470 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:03.286476 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 947/1883] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:02 2022] DELETE /identity/v3/users/19a07e583d404b658435a58898780279 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:03.307728 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 937/1884] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:45:03.317832 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fb8fc-2bc2-4f94-8c72-271e710aeb90 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:03.318333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fb8fc-2bc2-4f94-8c72-271e710aeb90 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:03.318670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fb8fc-2bc2-4f94-8c72-271e710aeb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:03.338414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-987fb8fc-2bc2-4f94-8c72-271e710aeb90 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:03.339696 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-987fb8fc-2bc2-4f94-8c72-271e710aeb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:03.399160 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 948/1885] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:03.403422 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee750f17-4dd0-4fc4-b8a6-653e40b573d9 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:03.413266 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee750f17-4dd0-4fc4-b8a6-653e40b573d9 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:03.417927 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c9bf8-db2e-445a-af7d-e6251c51693c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:03.418605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c9bf8-db2e-445a-af7d-e6251c51693c admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:03.418886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c9bf8-db2e-445a-af7d-e6251c51693c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:03.419195 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c9bf8-db2e-445a-af7d-e6251c51693c admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:03.427876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c9bf8-db2e-445a-af7d-e6251c51693c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:03.427876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c9bf8-db2e-445a-af7d-e6251c51693c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:03.427876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c9bf8-db2e-445a-af7d-e6251c51693c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:03.433367 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 938/1886] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:03.438795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fb8fc-2bc2-4f94-8c72-271e710aeb90 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:03.455218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fb8fc-2bc2-4f94-8c72-271e710aeb90 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:03.467034 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb197b2-e742-43c3-8307-4f05e41e09ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:03.468602 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb197b2-e742-43c3-8307-4f05e41e09ed admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:03.469258 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb197b2-e742-43c3-8307-4f05e41e09ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:03.470133 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb197b2-e742-43c3-8307-4f05e41e09ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:03.471515 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb197b2-e742-43c3-8307-4f05e41e09ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:03.474923 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb197b2-e742-43c3-8307-4f05e41e09ed admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:03.597243 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 949/1887] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:03 2022] POST /identity/v3/projects => generated 357 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:45:03.601415 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c9bf8-db2e-445a-af7d-e6251c51693c admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:03.612830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c9bf8-db2e-445a-af7d-e6251c51693c admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:03.681102 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cec41f-da32-45a3-9fac-57a1b170eb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:03.681812 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cec41f-da32-45a3-9fac-57a1b170eb84 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:03.682230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cec41f-da32-45a3-9fac-57a1b170eb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:03.682428 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cec41f-da32-45a3-9fac-57a1b170eb84 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:03.683270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cec41f-da32-45a3-9fac-57a1b170eb84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:03.685228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cec41f-da32-45a3-9fac-57a1b170eb84 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:03.791726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 939/1888] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:03 2022] POST /identity/v3/users => generated 279 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:03.792652 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb197b2-e742-43c3-8307-4f05e41e09ed admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:03.794553 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-aeb197b2-e742-43c3-8307-4f05e41e09ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:03.795354 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cec41f-da32-45a3-9fac-57a1b170eb84 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:03.807104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cec41f-da32-45a3-9fac-57a1b170eb84 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:03.812702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695c612-5885-4459-af7d-fc8176d098dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:03.813474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695c612-5885-4459-af7d-fc8176d098dd admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:03.813818 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb197b2-e742-43c3-8307-4f05e41e09ed admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:03.813930 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695c612-5885-4459-af7d-fc8176d098dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:03.814183 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695c612-5885-4459-af7d-fc8176d098dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:03.814939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695c612-5885-4459-af7d-fc8176d098dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:03.816538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695c612-5885-4459-af7d-fc8176d098dd admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:03.826065 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48e742-63ee-491e-a6ae-2deffbceabc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:03.827012 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48e742-63ee-491e-a6ae-2deffbceabc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:03.831231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 940/1889] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:45:03.831337 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48e742-63ee-491e-a6ae-2deffbceabc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:03.831337 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48e742-63ee-491e-a6ae-2deffbceabc7 admin admin] PATH_INFO: `/v3/projects/974513e8db8a46fc848c6a364adb7e73` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:03.831337 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48e742-63ee-491e-a6ae-2deffbceabc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=974513e8db8a46fc848c6a364adb7e73)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:03.831547 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48e742-63ee-491e-a6ae-2deffbceabc7 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:03.832447 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695c612-5885-4459-af7d-fc8176d098dd admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:03.846834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695c612-5885-4459-af7d-fc8176d098dd admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:03.852289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b0cd2-6cd3-4620-9e3f-d2fccac5f581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:03.852965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b0cd2-6cd3-4620-9e3f-d2fccac5f581 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:03.853251 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b0cd2-6cd3-4620-9e3f-d2fccac5f581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:03.853528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b0cd2-6cd3-4620-9e3f-d2fccac5f581 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:03.854267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b0cd2-6cd3-4620-9e3f-d2fccac5f581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:03.855925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b0cd2-6cd3-4620-9e3f-d2fccac5f581 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:03.868062 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 941/1890] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:45:03.872156 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b0cd2-6cd3-4620-9e3f-d2fccac5f581 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:03.882555 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b0cd2-6cd3-4620-9e3f-d2fccac5f581 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:03.887115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdbb19-866a-4f45-a50b-ffb1910fdefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:03.888144 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdbb19-866a-4f45-a50b-ffb1910fdefb admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:03.888643 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdbb19-866a-4f45-a50b-ffb1910fdefb admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:03.889113 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdbb19-866a-4f45-a50b-ffb1910fdefb admin admin] PATH_INFO: `/v3/projects/08cd9378c0374566a14006e6d977859d/users/495af1f316954583a0f1091b457917d1/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:03.904318 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bdbb19-866a-4f45-a50b-ffb1910fdefb admin admin] RBAC: Authorizing `identity:create_grant(project_id=08cd9378c0374566a14006e6d977859d, user_id=495af1f316954583a0f1091b457917d1, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:03.907116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bdbb19-866a-4f45-a50b-ffb1910fdefb admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:03.973452 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 942/1891] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:03 2022] PUT /identity/v3/projects/08cd9378c0374566a14006e6d977859d/users/495af1f316954583a0f1091b457917d1/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:45:03.981871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fe48e742-63ee-491e-a6ae-2deffbceabc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '974513e8db8a46fc848c6a364adb7e73', 'request_id': 'req-fe48e742-63ee-491e-a6ae-2deffbceabc7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:03.983334 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cc296-cf57-4924-9ed1-6f318908f99e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:03.983747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cc296-cf57-4924-9ed1-6f318908f99e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:03.984054 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cc296-cf57-4924-9ed1-6f318908f99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:03.987976 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fe48e742-63ee-491e-a6ae-2deffbceabc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '974513e8db8a46fc848c6a364adb7e73', 'request_id': 'req-fe48e742-63ee-491e-a6ae-2deffbceabc7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:04.010938 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-8b7cc296-cf57-4924-9ed1-6f318908f99e None None] MFA Rules not processed for user `495af1f316954583a0f1091b457917d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:04.012156 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-8b7cc296-cf57-4924-9ed1-6f318908f99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:04.079489 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fe48e742-63ee-491e-a6ae-2deffbceabc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '974513e8db8a46fc848c6a364adb7e73', 'request_id': 'req-fe48e742-63ee-491e-a6ae-2deffbceabc7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:04.079806 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 943/1892] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:03 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:45:04.095332 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:04.106109 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fe48e742-63ee-491e-a6ae-2deffbceabc7 admin admin] The token cache is being invalidate because project 974513e8db8a46fc848c6a364adb7e73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:04.106407 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fe48e742-63ee-491e-a6ae-2deffbceabc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:04.107196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fe48e742-63ee-491e-a6ae-2deffbceabc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:04.108861 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 950/1893] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:03 2022] DELETE /identity/v3/projects/974513e8db8a46fc848c6a364adb7e73 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:45:04.114557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a130625-c3c6-4e59-b9ac-bfa98e593d5e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:04.114926 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a130625-c3c6-4e59-b9ac-bfa98e593d5e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:04.115215 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a130625-c3c6-4e59-b9ac-bfa98e593d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:04.118234 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48e742-63ee-491e-a6ae-2deffbceabc7 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:04.119712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-fe48e742-63ee-491e-a6ae-2deffbceabc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:04.138089 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5a130625-c3c6-4e59-b9ac-bfa98e593d5e None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:04.145817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:04.150489 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5a130625-c3c6-4e59-b9ac-bfa98e593d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:04.194168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48e742-63ee-491e-a6ae-2deffbceabc7 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:04.198059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b96eb9-8a04-4911-9537-105267d8f8a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:04.198782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b96eb9-8a04-4911-9537-105267d8f8a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:04.199070 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b96eb9-8a04-4911-9537-105267d8f8a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:04.199379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b96eb9-8a04-4911-9537-105267d8f8a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:04.202459 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-74b96eb9-8a04-4911-9537-105267d8f8a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:04.215383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b96eb9-8a04-4911-9537-105267d8f8a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:04.216864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b96eb9-8a04-4911-9537-105267d8f8a5 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:04.234176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 944/1894] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:04.234351 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 951/1895] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:04 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:04.240406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57aa09-f1af-4be5-b33b-a457389da9fe None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:04.241321 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57aa09-f1af-4be5-b33b-a457389da9fe None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:04.241639 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57aa09-f1af-4be5-b33b-a457389da9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:04.246479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b96eb9-8a04-4911-9537-105267d8f8a5 service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:04.248022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-74b96eb9-8a04-4911-9537-105267d8f8a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:04.264454 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b96eb9-8a04-4911-9537-105267d8f8a5 service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:04.269747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-de57aa09-f1af-4be5-b33b-a457389da9fe None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:04.271065 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-de57aa09-f1af-4be5-b33b-a457389da9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:04.279030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fad972-6cb8-40d6-bd1b-1bdf7c3fd32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:04.280222 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fad972-6cb8-40d6-bd1b-1bdf7c3fd32f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:04.280572 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fad972-6cb8-40d6-bd1b-1bdf7c3fd32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:04.284100 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fad972-6cb8-40d6-bd1b-1bdf7c3fd32f admin admin] PATH_INFO: `/v3/users/495af1f316954583a0f1091b457917d1` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:04.284182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 945/1896] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:04.284856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fad972-6cb8-40d6-bd1b-1bdf7c3fd32f admin admin] RBAC: Authorizing `identity:delete_user(user_id=495af1f316954583a0f1091b457917d1)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:04.285580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57aa09-f1af-4be5-b33b-a457389da9fe None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:04.287303 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fad972-6cb8-40d6-bd1b-1bdf7c3fd32f admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:04.298009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57aa09-f1af-4be5-b33b-a457389da9fe None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:04.304921 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8725f5-32e0-42fe-9da1-3462532f6d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:04.308571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8725f5-32e0-42fe-9da1-3462532f6d8b admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:04.308571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8725f5-32e0-42fe-9da1-3462532f6d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:04.308571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8725f5-32e0-42fe-9da1-3462532f6d8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:04.308571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8725f5-32e0-42fe-9da1-3462532f6d8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:04.308571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8725f5-32e0-42fe-9da1-3462532f6d8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:04.309126 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8725f5-32e0-42fe-9da1-3462532f6d8b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:04.325797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 946/1897] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:04.344088 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8725f5-32e0-42fe-9da1-3462532f6d8b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:04.374424 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8725f5-32e0-42fe-9da1-3462532f6d8b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:04.379968 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261763ca-fa55-43bf-8c42-c74fbf84e9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:04.380687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261763ca-fa55-43bf-8c42-c74fbf84e9c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:04.380975 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261763ca-fa55-43bf-8c42-c74fbf84e9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:04.381252 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261763ca-fa55-43bf-8c42-c74fbf84e9c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:04.381976 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261763ca-fa55-43bf-8c42-c74fbf84e9c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:04.383967 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261763ca-fa55-43bf-8c42-c74fbf84e9c3 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:04.486127 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-30fad972-6cb8-40d6-bd1b-1bdf7c3fd32f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '495af1f316954583a0f1091b457917d1'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:04.498702 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-30fad972-6cb8-40d6-bd1b-1bdf7c3fd32f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '495af1f316954583a0f1091b457917d1'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:04.499433 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-30fad972-6cb8-40d6-bd1b-1bdf7c3fd32f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '495af1f316954583a0f1091b457917d1'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:04.500085 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-30fad972-6cb8-40d6-bd1b-1bdf7c3fd32f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '495af1f316954583a0f1091b457917d1'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:04.512932 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-30fad972-6cb8-40d6-bd1b-1bdf7c3fd32f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '495af1f316954583a0f1091b457917d1'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:04.517210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 947/1898] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:04 2022] POST /identity/v3/projects => generated 365 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:04.518231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261763ca-fa55-43bf-8c42-c74fbf84e9c3 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:04.519914 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-261763ca-fa55-43bf-8c42-c74fbf84e9c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:04.583570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261763ca-fa55-43bf-8c42-c74fbf84e9c3 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:04.588556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a18e57-52f9-4c0a-bf19-d2c114302d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:04.589261 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a18e57-52f9-4c0a-bf19-d2c114302d4d admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:04.589548 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a18e57-52f9-4c0a-bf19-d2c114302d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:04.589824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a18e57-52f9-4c0a-bf19-d2c114302d4d admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:04.590565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a18e57-52f9-4c0a-bf19-d2c114302d4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:04.592498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a18e57-52f9-4c0a-bf19-d2c114302d4d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:04.604547 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 952/1899] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:04 2022] DELETE /identity/v3/users/495af1f316954583a0f1091b457917d1 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:04.609797 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cca29-3528-4e59-a3fd-6ad5b20c74c8 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:04.611182 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cca29-3528-4e59-a3fd-6ad5b20c74c8 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:04.611538 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cca29-3528-4e59-a3fd-6ad5b20c74c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:04.634245 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-f56cca29-3528-4e59-a3fd-6ad5b20c74c8 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:04.635518 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f56cca29-3528-4e59-a3fd-6ad5b20c74c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:04.692303 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 953/1900] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:04.701989 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cca29-3528-4e59-a3fd-6ad5b20c74c8 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:04.703708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f56cca29-3528-4e59-a3fd-6ad5b20c74c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:04.708628 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 948/1901] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:04 2022] POST /identity/v3/users => generated 283 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:45:04.709965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a18e57-52f9-4c0a-bf19-d2c114302d4d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:04.720413 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a18e57-52f9-4c0a-bf19-d2c114302d4d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:04.729560 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74952c52-72e1-46b4-8155-e24e9c1ac067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:04.730234 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74952c52-72e1-46b4-8155-e24e9c1ac067 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:04.730627 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74952c52-72e1-46b4-8155-e24e9c1ac067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:04.730827 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74952c52-72e1-46b4-8155-e24e9c1ac067 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:04.731582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74952c52-72e1-46b4-8155-e24e9c1ac067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:04.733242 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74952c52-72e1-46b4-8155-e24e9c1ac067 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:04.747492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 949/1902] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:04.751967 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74952c52-72e1-46b4-8155-e24e9c1ac067 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:04.768249 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74952c52-72e1-46b4-8155-e24e9c1ac067 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:04.774700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd42b8-ac7f-413b-ac6e-8ea06ab53a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:04.775504 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd42b8-ac7f-413b-ac6e-8ea06ab53a5e admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:04.775859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd42b8-ac7f-413b-ac6e-8ea06ab53a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:04.776202 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd42b8-ac7f-413b-ac6e-8ea06ab53a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:04.777345 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd42b8-ac7f-413b-ac6e-8ea06ab53a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:04.779374 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd42b8-ac7f-413b-ac6e-8ea06ab53a5e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:04.783567 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:04.793404 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cca29-3528-4e59-a3fd-6ad5b20c74c8 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:04.793620 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 950/1903] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:04.797190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd42b8-ac7f-413b-ac6e-8ea06ab53a5e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:04.797403 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23dc3b1-12d8-452f-9ff8-068c78048113 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:04.798940 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23dc3b1-12d8-452f-9ff8-068c78048113 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:04.799590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23dc3b1-12d8-452f-9ff8-068c78048113 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:04.800739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23dc3b1-12d8-452f-9ff8-068c78048113 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:04.811539 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23dc3b1-12d8-452f-9ff8-068c78048113 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:04.811965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd42b8-ac7f-413b-ac6e-8ea06ab53a5e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:04.813376 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23dc3b1-12d8-452f-9ff8-068c78048113 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:04.817701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b4ad9-3a7e-4bd1-8bc6-9301da4f4c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:04.818354 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b4ad9-3a7e-4bd1-8bc6-9301da4f4c36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:04.818807 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b4ad9-3a7e-4bd1-8bc6-9301da4f4c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:04.819161 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b4ad9-3a7e-4bd1-8bc6-9301da4f4c36 admin admin] PATH_INFO: `/v3/projects/267edba984644887b4b64b816f2d8a1d/users/ae41c7b514b84689b97b6912deeec5a3/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:04.821352 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:04.838005 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 954/1904] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:45:04.838316 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b4ad9-3a7e-4bd1-8bc6-9301da4f4c36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=267edba984644887b4b64b816f2d8a1d, user_id=ae41c7b514b84689b97b6912deeec5a3, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:04.841076 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b4ad9-3a7e-4bd1-8bc6-9301da4f4c36 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:04.906578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 951/1905] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:04 2022] PUT /identity/v3/projects/267edba984644887b4b64b816f2d8a1d/users/ae41c7b514b84689b97b6912deeec5a3/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:45:04.918105 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e6d98-a8bb-43b4-a443-935362f378a6 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:04.918508 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e6d98-a8bb-43b4-a443-935362f378a6 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:04.918956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e6d98-a8bb-43b4-a443-935362f378a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:04.947900 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-a57e6d98-a8bb-43b4-a443-935362f378a6 None None] MFA Rules not processed for user `ae41c7b514b84689b97b6912deeec5a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:04.949220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a57e6d98-a8bb-43b4-a443-935362f378a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:05.017008 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 955/1906] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:04 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:05.021428 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b4ad9-3a7e-4bd1-8bc6-9301da4f4c36 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:05.094256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b4ad9-3a7e-4bd1-8bc6-9301da4f4c36 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:05.099389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949481a-267e-4cb7-9eca-8a7c62f188c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:05.100054 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949481a-267e-4cb7-9eca-8a7c62f188c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:05.100339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949481a-267e-4cb7-9eca-8a7c62f188c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:05.100617 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949481a-267e-4cb7-9eca-8a7c62f188c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:05.101315 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949481a-267e-4cb7-9eca-8a7c62f188c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:05.103273 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949481a-267e-4cb7-9eca-8a7c62f188c4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:05.199245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 952/1907] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:05 2022] POST /identity/v3/projects => generated 361 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:45:05.203217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e6d98-a8bb-43b4-a443-935362f378a6 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:05.214235 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e6d98-a8bb-43b4-a443-935362f378a6 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:05.255498 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:05.264655 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949481a-267e-4cb7-9eca-8a7c62f188c4 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:05.264655 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2949481a-267e-4cb7-9eca-8a7c62f188c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:05.280310 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a5a61-1df6-4501-a98d-158606e72e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:05.281040 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a5a61-1df6-4501-a98d-158606e72e3d admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:05.281336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a5a61-1df6-4501-a98d-158606e72e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:05.281603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a5a61-1df6-4501-a98d-158606e72e3d admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:05.283047 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a5a61-1df6-4501-a98d-158606e72e3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:05.285948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a5a61-1df6-4501-a98d-158606e72e3d admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:05.344616 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949481a-267e-4cb7-9eca-8a7c62f188c4 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:05.349628 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8babb79a-3a40-42e0-87f3-cac2bd46d77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:05.350493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babb79a-3a40-42e0-87f3-cac2bd46d77c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:05.350804 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babb79a-3a40-42e0-87f3-cac2bd46d77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:05.351073 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babb79a-3a40-42e0-87f3-cac2bd46d77c admin admin] PATH_INFO: `/v3/projects/08cd9378c0374566a14006e6d977859d` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:05.352438 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8babb79a-3a40-42e0-87f3-cac2bd46d77c admin admin] RBAC: Authorizing `identity:delete_project(project_id=08cd9378c0374566a14006e6d977859d)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:05.354174 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8babb79a-3a40-42e0-87f3-cac2bd46d77c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:05.378445 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 956/1908] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:05 2022] POST /identity/v3/users => generated 287 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:05.382231 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a5a61-1df6-4501-a98d-158606e72e3d admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:05.399260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a5a61-1df6-4501-a98d-158606e72e3d admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:05.410676 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c31bbf-2bc0-4f3b-9e05-19474cd0d43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:05.411482 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c31bbf-2bc0-4f3b-9e05-19474cd0d43f admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:05.411849 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c31bbf-2bc0-4f3b-9e05-19474cd0d43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:05.412204 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c31bbf-2bc0-4f3b-9e05-19474cd0d43f admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:05.413206 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c31bbf-2bc0-4f3b-9e05-19474cd0d43f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:05.415291 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c31bbf-2bc0-4f3b-9e05-19474cd0d43f admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:05.429751 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 957/1909] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:05.432668 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c31bbf-2bc0-4f3b-9e05-19474cd0d43f admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:05.450406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c31bbf-2bc0-4f3b-9e05-19474cd0d43f admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:05.457289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e66c9-b05d-455b-a055-942e353684ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:05.458209 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e66c9-b05d-455b-a055-942e353684ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:05.458812 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e66c9-b05d-455b-a055-942e353684ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:05.459505 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e66c9-b05d-455b-a055-942e353684ad admin admin] PATH_INFO: `/v3/projects/e47e8523e462478cabad7b31a5fa9999/users/f506fd764813480fbe991a6b93df117f/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:05.479418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e66c9-b05d-455b-a055-942e353684ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=e47e8523e462478cabad7b31a5fa9999, user_id=f506fd764813480fbe991a6b93df117f, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:05.486100 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e66c9-b05d-455b-a055-942e353684ad admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:05.590886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 958/1910] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:05 2022] PUT /identity/v3/projects/e47e8523e462478cabad7b31a5fa9999/users/f506fd764813480fbe991a6b93df117f/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:05.595639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e66c9-b05d-455b-a055-942e353684ad admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:05.604833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8babb79a-3a40-42e0-87f3-cac2bd46d77c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08cd9378c0374566a14006e6d977859d', 'request_id': 'req-8babb79a-3a40-42e0-87f3-cac2bd46d77c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:05.616635 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8babb79a-3a40-42e0-87f3-cac2bd46d77c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08cd9378c0374566a14006e6d977859d', 'request_id': 'req-8babb79a-3a40-42e0-87f3-cac2bd46d77c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:05.692121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e66c9-b05d-455b-a055-942e353684ad admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:05.694821 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8babb79a-3a40-42e0-87f3-cac2bd46d77c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08cd9378c0374566a14006e6d977859d', 'request_id': 'req-8babb79a-3a40-42e0-87f3-cac2bd46d77c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:05.696314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:05.706942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8babb79a-3a40-42e0-87f3-cac2bd46d77c admin admin] The token cache is being invalidate because project 08cd9378c0374566a14006e6d977859d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:05.707225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8babb79a-3a40-42e0-87f3-cac2bd46d77c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:05.707852 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8babb79a-3a40-42e0-87f3-cac2bd46d77c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:05.709901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 953/1911] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:05 2022] DELETE /identity/v3/projects/08cd9378c0374566a14006e6d977859d => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:45:05.714368 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb12d9-d5a8-4477-b785-724c403f69db None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:05.714693 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb12d9-d5a8-4477-b785-724c403f69db None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:05.714977 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb12d9-d5a8-4477-b785-724c403f69db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:05.733801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-00fb12d9-d5a8-4477-b785-724c403f69db None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:05.735085 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-00fb12d9-d5a8-4477-b785-724c403f69db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:05.755466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb246245-cc08-43f1-b357-24a6adbccb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:05.756336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb246245-cc08-43f1-b357-24a6adbccb29 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:05.756693 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb246245-cc08-43f1-b357-24a6adbccb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:05.757041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb246245-cc08-43f1-b357-24a6adbccb29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:05.757837 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb246245-cc08-43f1-b357-24a6adbccb29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:05.759528 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb246245-cc08-43f1-b357-24a6adbccb29 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:05.777564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 959/1912] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:05.778523 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb246245-cc08-43f1-b357-24a6adbccb29 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:05.780354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-fb246245-cc08-43f1-b357-24a6adbccb29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:05.795012 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb246245-cc08-43f1-b357-24a6adbccb29 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:05.799956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0c5c7-6645-46e6-962d-c7e8b2477abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:05.800996 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0c5c7-6645-46e6-962d-c7e8b2477abf admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:05.801288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0c5c7-6645-46e6-962d-c7e8b2477abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:05.801577 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0c5c7-6645-46e6-962d-c7e8b2477abf admin admin] PATH_INFO: `/v3/projects/e47e8523e462478cabad7b31a5fa9999/users/f506fd764813480fbe991a6b93df117f/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:05.803969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0c5c7-6645-46e6-962d-c7e8b2477abf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e47e8523e462478cabad7b31a5fa9999, user_id=f506fd764813480fbe991a6b93df117f, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:05.804456 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 954/1913] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:45:05.809187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0c5c7-6645-46e6-962d-c7e8b2477abf admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:05.813979 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df58cd-21e2-4b02-bbcf-4dff304ca242 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:05.815032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df58cd-21e2-4b02-bbcf-4dff304ca242 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:05.815554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df58cd-21e2-4b02-bbcf-4dff304ca242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:05.829374 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 960/1914] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:05 2022] PUT /identity/v3/projects/e47e8523e462478cabad7b31a5fa9999/users/f506fd764813480fbe991a6b93df117f/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:45:05.841003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07355dde-f6be-42da-8d3c-fa1462a10e71 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:05.841296 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07355dde-f6be-42da-8d3c-fa1462a10e71 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:05.841574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07355dde-f6be-42da-8d3c-fa1462a10e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:05.853676 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f9df58cd-21e2-4b02-bbcf-4dff304ca242 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:05.854965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f9df58cd-21e2-4b02-bbcf-4dff304ca242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:05.868661 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-07355dde-f6be-42da-8d3c-fa1462a10e71 None None] MFA Rules not processed for user `f506fd764813480fbe991a6b93df117f`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:05.869848 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-07355dde-f6be-42da-8d3c-fa1462a10e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:05.882885 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:05.918262 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 955/1915] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:45:05.922664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df58cd-21e2-4b02-bbcf-4dff304ca242 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:05.931420 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df58cd-21e2-4b02-bbcf-4dff304ca242 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:05.936039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb27268-fa1d-494b-b9e4-ba9016ee0d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:05.936711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb27268-fa1d-494b-b9e4-ba9016ee0d57 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:05.937014 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb27268-fa1d-494b-b9e4-ba9016ee0d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:05.937288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb27268-fa1d-494b-b9e4-ba9016ee0d57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:05.938138 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb27268-fa1d-494b-b9e4-ba9016ee0d57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:05.938353 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 961/1916] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:05 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:45:05.938451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb27268-fa1d-494b-b9e4-ba9016ee0d57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:05.940215 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb27268-fa1d-494b-b9e4-ba9016ee0d57 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:05.950227 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07355dde-f6be-42da-8d3c-fa1462a10e71 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:05.951619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-07355dde-f6be-42da-8d3c-fa1462a10e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:05.956081 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 956/1917] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:45:05.957089 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb27268-fa1d-494b-b9e4-ba9016ee0d57 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:05.967262 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb27268-fa1d-494b-b9e4-ba9016ee0d57 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:05.973718 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b5236-ba34-433b-b1a7-830d90c50cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:05.974811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b5236-ba34-433b-b1a7-830d90c50cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:05.975102 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b5236-ba34-433b-b1a7-830d90c50cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:05.975382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b5236-ba34-433b-b1a7-830d90c50cc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:05.976391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b5236-ba34-433b-b1a7-830d90c50cc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:05.978657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b5236-ba34-433b-b1a7-830d90c50cc5 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:06.049077 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07355dde-f6be-42da-8d3c-fa1462a10e71 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:06.049077 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fe0ba-2dd3-495b-becb-5ab84f5a1fef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:06.049077 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fe0ba-2dd3-495b-becb-5ab84f5a1fef service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:06.049077 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fe0ba-2dd3-495b-becb-5ab84f5a1fef service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:06.049498 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fe0ba-2dd3-495b-becb-5ab84f5a1fef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:06.054190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fe0ba-2dd3-495b-becb-5ab84f5a1fef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:06.055698 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fe0ba-2dd3-495b-becb-5ab84f5a1fef service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:06.065879 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 962/1918] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:05 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:45:06.081820 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 957/1919] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:05 2022] POST /identity/v3/projects => generated 363 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:06.085996 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fe0ba-2dd3-495b-becb-5ab84f5a1fef service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:06.099213 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fe0ba-2dd3-495b-becb-5ab84f5a1fef service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:06.167293 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e05ce-1da2-4d35-8608-77d2e9e7641d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:06.168006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e05ce-1da2-4d35-8608-77d2e9e7641d admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:06.168296 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e05ce-1da2-4d35-8608-77d2e9e7641d admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:06.168599 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e05ce-1da2-4d35-8608-77d2e9e7641d admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:06.169359 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e05ce-1da2-4d35-8608-77d2e9e7641d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:06.171485 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e05ce-1da2-4d35-8608-77d2e9e7641d admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:06.293228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 963/1920] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:06 2022] POST /identity/v3/users => generated 282 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:06.297342 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b5236-ba34-433b-b1a7-830d90c50cc5 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:06.308093 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b5236-ba34-433b-b1a7-830d90c50cc5 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:06.315328 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63677f6-510f-4172-b083-94a171f6127a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:06.315995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63677f6-510f-4172-b083-94a171f6127a admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:06.316280 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63677f6-510f-4172-b083-94a171f6127a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:06.316582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63677f6-510f-4172-b083-94a171f6127a admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:06.317362 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63677f6-510f-4172-b083-94a171f6127a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:06.319091 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63677f6-510f-4172-b083-94a171f6127a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:06.330204 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 958/1921] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:06.339023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e05ce-1da2-4d35-8608-77d2e9e7641d admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:06.349235 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e05ce-1da2-4d35-8608-77d2e9e7641d admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:06.353979 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66281fdc-4cab-482d-8a47-fbe3991206a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:06.354663 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66281fdc-4cab-482d-8a47-fbe3991206a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:06.354943 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66281fdc-4cab-482d-8a47-fbe3991206a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:06.355212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66281fdc-4cab-482d-8a47-fbe3991206a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:06.355965 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66281fdc-4cab-482d-8a47-fbe3991206a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:06.357530 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66281fdc-4cab-482d-8a47-fbe3991206a6 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:06.370124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 964/1922] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:06.378228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63677f6-510f-4172-b083-94a171f6127a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:06.379442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66281fdc-4cab-482d-8a47-fbe3991206a6 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:06.389198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66281fdc-4cab-482d-8a47-fbe3991206a6 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:06.395042 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a2e12-50fd-4976-b1e8-536030dac1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:06.396001 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a2e12-50fd-4976-b1e8-536030dac1ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:06.396281 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a2e12-50fd-4976-b1e8-536030dac1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:06.396552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a2e12-50fd-4976-b1e8-536030dac1ee admin admin] PATH_INFO: `/v3/projects/1508ad150f004c739d9f681c81648268/users/fc5b5e637c9849fcb05a669133eef001/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:06.414031 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a2e12-50fd-4976-b1e8-536030dac1ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=1508ad150f004c739d9f681c81648268, user_id=fc5b5e637c9849fcb05a669133eef001, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:06.416961 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a2e12-50fd-4976-b1e8-536030dac1ee admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:06.428554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:06.438743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63677f6-510f-4172-b083-94a171f6127a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:06.442501 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eac6d2-3a2c-4948-a26c-1b5a5112634b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:06.443189 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eac6d2-3a2c-4948-a26c-1b5a5112634b service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:06.443474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eac6d2-3a2c-4948-a26c-1b5a5112634b service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:06.443756 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eac6d2-3a2c-4948-a26c-1b5a5112634b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:06.445611 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d0eac6d2-3a2c-4948-a26c-1b5a5112634b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:06.496751 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 965/1923] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:06 2022] PUT /identity/v3/projects/1508ad150f004c739d9f681c81648268/users/fc5b5e637c9849fcb05a669133eef001/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:06.507543 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96140c03-3026-4854-a6a2-bea545398079 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:06.507836 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96140c03-3026-4854-a6a2-bea545398079 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:06.508135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96140c03-3026-4854-a6a2-bea545398079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:06.522960 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eac6d2-3a2c-4948-a26c-1b5a5112634b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:06.524393 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eac6d2-3a2c-4948-a26c-1b5a5112634b service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:06.526474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:06.533373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 959/1924] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:06 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:06.535242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-96140c03-3026-4854-a6a2-bea545398079 None None] MFA Rules not processed for user `fc5b5e637c9849fcb05a669133eef001`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:06.536443 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-96140c03-3026-4854-a6a2-bea545398079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:06.600861 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 966/1925] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:06 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:45:06.612553 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eac6d2-3a2c-4948-a26c-1b5a5112634b service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:06.613989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d0eac6d2-3a2c-4948-a26c-1b5a5112634b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:06.624798 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eac6d2-3a2c-4948-a26c-1b5a5112634b service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:06.679677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276279b2-8893-41fc-b3f7-779785486f85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:06.680373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276279b2-8893-41fc-b3f7-779785486f85 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:06.680615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276279b2-8893-41fc-b3f7-779785486f85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:06.680892 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276279b2-8893-41fc-b3f7-779785486f85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:06.690846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276279b2-8893-41fc-b3f7-779785486f85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:06.692366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276279b2-8893-41fc-b3f7-779785486f85 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:06.701672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 960/1926] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:06 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:10.890717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96140c03-3026-4854-a6a2-bea545398079 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:10.967563 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96140c03-3026-4854-a6a2-bea545398079 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:10.972607 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2beba-f8f4-46a6-a692-f19b1244f47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:10.973464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2beba-f8f4-46a6-a692-f19b1244f47b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:10.973747 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2beba-f8f4-46a6-a692-f19b1244f47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:10.974016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2beba-f8f4-46a6-a692-f19b1244f47b admin admin] PATH_INFO: `/v3/users/ae41c7b514b84689b97b6912deeec5a3` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:10.975275 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2beba-f8f4-46a6-a692-f19b1244f47b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae41c7b514b84689b97b6912deeec5a3)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:10.977172 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2beba-f8f4-46a6-a692-f19b1244f47b admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:11.190974 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-45e2beba-f8f4-46a6-a692-f19b1244f47b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae41c7b514b84689b97b6912deeec5a3'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:11.199130 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-45e2beba-f8f4-46a6-a692-f19b1244f47b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae41c7b514b84689b97b6912deeec5a3'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:11.199876 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-45e2beba-f8f4-46a6-a692-f19b1244f47b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae41c7b514b84689b97b6912deeec5a3'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:11.200471 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-45e2beba-f8f4-46a6-a692-f19b1244f47b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae41c7b514b84689b97b6912deeec5a3'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:11.213688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-45e2beba-f8f4-46a6-a692-f19b1244f47b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae41c7b514b84689b97b6912deeec5a3'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:11.282838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 967/1927] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:10 2022] DELETE /identity/v3/users/ae41c7b514b84689b97b6912deeec5a3 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:11.288198 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ba989-809e-4ef1-8ea6-6607e378b74d None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:11.288701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ba989-809e-4ef1-8ea6-6607e378b74d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:11.288820 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ba989-809e-4ef1-8ea6-6607e378b74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:11.306429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b19ba989-809e-4ef1-8ea6-6607e378b74d None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:11.307636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b19ba989-809e-4ef1-8ea6-6607e378b74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:11.372315 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 961/1928] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:11.386045 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2beba-f8f4-46a6-a692-f19b1244f47b admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:11.387521 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-45e2beba-f8f4-46a6-a692-f19b1244f47b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:11.412030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:11.451833 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2beba-f8f4-46a6-a692-f19b1244f47b admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:11.456175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9ebc0-0770-449d-9d7f-0563516bb5dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:11.457821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9ebc0-0770-449d-9d7f-0563516bb5dd service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:11.458114 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9ebc0-0770-449d-9d7f-0563516bb5dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:11.458599 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9ebc0-0770-449d-9d7f-0563516bb5dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:11.476942 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9ebc0-0770-449d-9d7f-0563516bb5dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:11.482243 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9ebc0-0770-449d-9d7f-0563516bb5dd service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:11.499673 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 968/1929] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:12.111315 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ba989-809e-4ef1-8ea6-6607e378b74d None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:12.113843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b19ba989-809e-4ef1-8ea6-6607e378b74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:12.127216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ba989-809e-4ef1-8ea6-6607e378b74d None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:12.132537 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e59ed-352f-46d1-9975-73b738ef84b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:12.133707 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e59ed-352f-46d1-9975-73b738ef84b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:12.134213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e59ed-352f-46d1-9975-73b738ef84b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:12.134567 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e59ed-352f-46d1-9975-73b738ef84b8 admin admin] PATH_INFO: `/v3/projects/267edba984644887b4b64b816f2d8a1d` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:12.136020 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e59ed-352f-46d1-9975-73b738ef84b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=267edba984644887b4b64b816f2d8a1d)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:12.138362 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e59ed-352f-46d1-9975-73b738ef84b8 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:12.273148 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:12.281167 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7d5e59ed-352f-46d1-9975-73b738ef84b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '267edba984644887b4b64b816f2d8a1d', 'request_id': 'req-7d5e59ed-352f-46d1-9975-73b738ef84b8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:12.287851 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7d5e59ed-352f-46d1-9975-73b738ef84b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '267edba984644887b4b64b816f2d8a1d', 'request_id': 'req-7d5e59ed-352f-46d1-9975-73b738ef84b8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:12.403009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7d5e59ed-352f-46d1-9975-73b738ef84b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '267edba984644887b4b64b816f2d8a1d', 'request_id': 'req-7d5e59ed-352f-46d1-9975-73b738ef84b8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:12.414374 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7d5e59ed-352f-46d1-9975-73b738ef84b8 admin admin] The token cache is being invalidate because project 267edba984644887b4b64b816f2d8a1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:12.414690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7d5e59ed-352f-46d1-9975-73b738ef84b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:12.415315 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7d5e59ed-352f-46d1-9975-73b738ef84b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:12.416888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 962/1930] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:12 2022] DELETE /identity/v3/projects/267edba984644887b4b64b816f2d8a1d => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:12.420904 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9ebc0-0770-449d-9d7f-0563516bb5dd service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:12.430861 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9ebc0-0770-449d-9d7f-0563516bb5dd service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:12.491253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645816be-a799-4163-afd1-8a5ed07d602a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:12.492130 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645816be-a799-4163-afd1-8a5ed07d602a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:12.492669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645816be-a799-4163-afd1-8a5ed07d602a admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:12.493104 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645816be-a799-4163-afd1-8a5ed07d602a admin admin] PATH_INFO: `/v3/users/f506fd764813480fbe991a6b93df117f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:12.494375 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645816be-a799-4163-afd1-8a5ed07d602a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f506fd764813480fbe991a6b93df117f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:12.496335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645816be-a799-4163-afd1-8a5ed07d602a admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:12.697889 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-645816be-a799-4163-afd1-8a5ed07d602a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f506fd764813480fbe991a6b93df117f'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:12.705229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-645816be-a799-4163-afd1-8a5ed07d602a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f506fd764813480fbe991a6b93df117f'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:12.706840 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-645816be-a799-4163-afd1-8a5ed07d602a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f506fd764813480fbe991a6b93df117f'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:12.707400 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-645816be-a799-4163-afd1-8a5ed07d602a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f506fd764813480fbe991a6b93df117f'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:12.720506 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-645816be-a799-4163-afd1-8a5ed07d602a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f506fd764813480fbe991a6b93df117f'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:12.833079 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 969/1931] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:12 2022] DELETE /identity/v3/users/f506fd764813480fbe991a6b93df117f => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:13.328586 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e59ed-352f-46d1-9975-73b738ef84b8 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:13.329995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7d5e59ed-352f-46d1-9975-73b738ef84b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:13.343482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e59ed-352f-46d1-9975-73b738ef84b8 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:13.402162 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad58f3-be4b-4086-a709-386078833ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:13.403168 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad58f3-be4b-4086-a709-386078833ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:13.403727 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad58f3-be4b-4086-a709-386078833ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:13.403727 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad58f3-be4b-4086-a709-386078833ca0 admin admin] PATH_INFO: `/v3/projects/e47e8523e462478cabad7b31a5fa9999` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:13.404913 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad58f3-be4b-4086-a709-386078833ca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e47e8523e462478cabad7b31a5fa9999)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:13.406908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad58f3-be4b-4086-a709-386078833ca0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:13.507561 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f6ad58f3-be4b-4086-a709-386078833ca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e47e8523e462478cabad7b31a5fa9999', 'request_id': 'req-f6ad58f3-be4b-4086-a709-386078833ca0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:13.513645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f6ad58f3-be4b-4086-a709-386078833ca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e47e8523e462478cabad7b31a5fa9999', 'request_id': 'req-f6ad58f3-be4b-4086-a709-386078833ca0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:13.579802 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f6ad58f3-be4b-4086-a709-386078833ca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e47e8523e462478cabad7b31a5fa9999', 'request_id': 'req-f6ad58f3-be4b-4086-a709-386078833ca0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:13.591349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f6ad58f3-be4b-4086-a709-386078833ca0 admin admin] The token cache is being invalidate because project e47e8523e462478cabad7b31a5fa9999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:13.591696 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f6ad58f3-be4b-4086-a709-386078833ca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:13.592402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f6ad58f3-be4b-4086-a709-386078833ca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:13.593989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 963/1932] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:13 2022] DELETE /identity/v3/projects/e47e8523e462478cabad7b31a5fa9999 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:13.599776 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12dee4-b87b-4ed5-bcc9-d4c712e64ccb None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:13.600232 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12dee4-b87b-4ed5-bcc9-d4c712e64ccb None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:13.600335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12dee4-b87b-4ed5-bcc9-d4c712e64ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:13.619303 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-0f12dee4-b87b-4ed5-bcc9-d4c712e64ccb None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:13.620479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-0f12dee4-b87b-4ed5-bcc9-d4c712e64ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:13.653476 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:13.679040 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 970/1933] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:13.686836 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd829713-2e4c-4f04-9038-533b9afe9dc5 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:13.687123 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd829713-2e4c-4f04-9038-533b9afe9dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:13.687578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd829713-2e4c-4f04-9038-533b9afe9dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:13.706239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-bd829713-2e4c-4f04-9038-533b9afe9dc5 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:13.707456 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bd829713-2e4c-4f04-9038-533b9afe9dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:13.716067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 964/1934] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:13.720112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f12dee4-b87b-4ed5-bcc9-d4c712e64ccb None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:13.729999 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f12dee4-b87b-4ed5-bcc9-d4c712e64ccb None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:13.734548 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe2b5a-ed9f-4077-a0f9-da46ff9f995b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:13.735204 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe2b5a-ed9f-4077-a0f9-da46ff9f995b admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:13.735487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe2b5a-ed9f-4077-a0f9-da46ff9f995b admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:13.735762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe2b5a-ed9f-4077-a0f9-da46ff9f995b admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:13.742142 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe2b5a-ed9f-4077-a0f9-da46ff9f995b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:13.742142 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe2b5a-ed9f-4077-a0f9-da46ff9f995b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:13.742142 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe2b5a-ed9f-4077-a0f9-da46ff9f995b admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:13.751423 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 971/1935] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:13.755709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd829713-2e4c-4f04-9038-533b9afe9dc5 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:13.765938 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd829713-2e4c-4f04-9038-533b9afe9dc5 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:13.771254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843bef7e-47f6-415c-96f7-13900c2ab29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:13.771929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bef7e-47f6-415c-96f7-13900c2ab29a admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:13.772216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bef7e-47f6-415c-96f7-13900c2ab29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:13.772492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bef7e-47f6-415c-96f7-13900c2ab29a admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:13.773298 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843bef7e-47f6-415c-96f7-13900c2ab29a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:13.775600 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843bef7e-47f6-415c-96f7-13900c2ab29a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:14.045914 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 965/1936] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:13 2022] POST /identity/v3/projects => generated 367 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:14.070622 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe2b5a-ed9f-4077-a0f9-da46ff9f995b admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:14.080573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe2b5a-ed9f-4077-a0f9-da46ff9f995b admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:14.136950 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc87ec-7fbe-4ac2-a513-6fae40ec79e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:14.137618 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc87ec-7fbe-4ac2-a513-6fae40ec79e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:14.137901 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc87ec-7fbe-4ac2-a513-6fae40ec79e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:14.138175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc87ec-7fbe-4ac2-a513-6fae40ec79e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:14.138891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc87ec-7fbe-4ac2-a513-6fae40ec79e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:14.140794 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc87ec-7fbe-4ac2-a513-6fae40ec79e2 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:14.373405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 972/1937] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:14 2022] POST /identity/v3/users => generated 284 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:14.379340 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843bef7e-47f6-415c-96f7-13900c2ab29a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:14.390397 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843bef7e-47f6-415c-96f7-13900c2ab29a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:14.396315 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd835e-2e11-4ce8-8e59-2caf54cca729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:14.397033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd835e-2e11-4ce8-8e59-2caf54cca729 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:14.397318 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd835e-2e11-4ce8-8e59-2caf54cca729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:14.397597 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd835e-2e11-4ce8-8e59-2caf54cca729 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:14.398336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd835e-2e11-4ce8-8e59-2caf54cca729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:14.400211 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd835e-2e11-4ce8-8e59-2caf54cca729 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:14.415760 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 966/1938] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:14.416950 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc87ec-7fbe-4ac2-a513-6fae40ec79e2 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:14.427796 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc87ec-7fbe-4ac2-a513-6fae40ec79e2 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:14.432825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329bcc2d-d73d-4fac-819e-ef9b5770732b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:14.433508 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bcc2d-d73d-4fac-819e-ef9b5770732b admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:14.433793 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bcc2d-d73d-4fac-819e-ef9b5770732b admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:14.434078 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bcc2d-d73d-4fac-819e-ef9b5770732b admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:14.434889 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329bcc2d-d73d-4fac-819e-ef9b5770732b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:14.436538 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329bcc2d-d73d-4fac-819e-ef9b5770732b admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:14.448434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 973/1939] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:45:14.452888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd835e-2e11-4ce8-8e59-2caf54cca729 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:14.464547 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd835e-2e11-4ce8-8e59-2caf54cca729 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:14.469206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff7356-35c6-4f6a-99b1-eff7b2fba75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:14.470107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff7356-35c6-4f6a-99b1-eff7b2fba75c admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:14.470390 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff7356-35c6-4f6a-99b1-eff7b2fba75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:14.470706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff7356-35c6-4f6a-99b1-eff7b2fba75c admin admin] PATH_INFO: `/v3/projects/d655240a40b3461f9ea914427a8a70aa/users/3b89fbff0a3d4b1b970de6a8138b93bf/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:14.487666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:14.488323 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff7356-35c6-4f6a-99b1-eff7b2fba75c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d655240a40b3461f9ea914427a8a70aa, user_id=3b89fbff0a3d4b1b970de6a8138b93bf, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:14.491217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff7356-35c6-4f6a-99b1-eff7b2fba75c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:14.592199 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 967/1940] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:14 2022] PUT /identity/v3/projects/d655240a40b3461f9ea914427a8a70aa/users/3b89fbff0a3d4b1b970de6a8138b93bf/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:14.617259 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ac048-6375-41eb-8742-ada99f0194d1 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:14.617800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ac048-6375-41eb-8742-ada99f0194d1 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:14.617800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ac048-6375-41eb-8742-ada99f0194d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:14.645117 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-d10ac048-6375-41eb-8742-ada99f0194d1 None None] MFA Rules not processed for user `3b89fbff0a3d4b1b970de6a8138b93bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:14.648087 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d10ac048-6375-41eb-8742-ada99f0194d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:14.721013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 974/1941] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:14 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:14.725840 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff7356-35c6-4f6a-99b1-eff7b2fba75c admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:14.787216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff7356-35c6-4f6a-99b1-eff7b2fba75c admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:14.791833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65531d67-044c-4bab-a1c1-78fbefc87c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:14.792487 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65531d67-044c-4bab-a1c1-78fbefc87c51 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:14.792771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65531d67-044c-4bab-a1c1-78fbefc87c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:14.793046 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65531d67-044c-4bab-a1c1-78fbefc87c51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:14.793740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65531d67-044c-4bab-a1c1-78fbefc87c51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:14.795676 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65531d67-044c-4bab-a1c1-78fbefc87c51 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:14.895381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 968/1942] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:14 2022] POST /identity/v3/projects => generated 369 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:14.903216 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ac048-6375-41eb-8742-ada99f0194d1 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:14.907670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ac048-6375-41eb-8742-ada99f0194d1 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:14.939455 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:14.963793 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef435a-38fb-48c1-b8ea-6f3fc23dfe70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:14.964464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef435a-38fb-48c1-b8ea-6f3fc23dfe70 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:14.964750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef435a-38fb-48c1-b8ea-6f3fc23dfe70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:14.965029 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef435a-38fb-48c1-b8ea-6f3fc23dfe70 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:14.965739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef435a-38fb-48c1-b8ea-6f3fc23dfe70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:14.967717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef435a-38fb-48c1-b8ea-6f3fc23dfe70 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:15.088103 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 975/1943] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:14 2022] POST /identity/v3/users => generated 291 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:15.089095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65531d67-044c-4bab-a1c1-78fbefc87c51 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:15.101480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65531d67-044c-4bab-a1c1-78fbefc87c51 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:15.106169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a46a66-dbc3-4037-b7a0-54af91ea2b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:15.106881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a46a66-dbc3-4037-b7a0-54af91ea2b19 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:15.107162 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a46a66-dbc3-4037-b7a0-54af91ea2b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:15.107435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a46a66-dbc3-4037-b7a0-54af91ea2b19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:15.108156 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a46a66-dbc3-4037-b7a0-54af91ea2b19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:15.110075 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a46a66-dbc3-4037-b7a0-54af91ea2b19 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:15.123209 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 969/1944] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:15.127620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef435a-38fb-48c1-b8ea-6f3fc23dfe70 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:15.137740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef435a-38fb-48c1-b8ea-6f3fc23dfe70 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:15.142368 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de98cb-7edc-44bd-8c6a-15fd4f553bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:15.143287 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de98cb-7edc-44bd-8c6a-15fd4f553bb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:15.143571 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de98cb-7edc-44bd-8c6a-15fd4f553bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:15.143916 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de98cb-7edc-44bd-8c6a-15fd4f553bb7 admin admin] PATH_INFO: `/v3/projects/fddd17237117482e8864620da2a460a0/users/e1ec9f2cc17b4176b65512e701eeabac/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:15.158734 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de98cb-7edc-44bd-8c6a-15fd4f553bb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fddd17237117482e8864620da2a460a0, user_id=e1ec9f2cc17b4176b65512e701eeabac, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:15.161940 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de98cb-7edc-44bd-8c6a-15fd4f553bb7 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:15.230449 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 976/1945] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:15 2022] PUT /identity/v3/projects/fddd17237117482e8864620da2a460a0/users/e1ec9f2cc17b4176b65512e701eeabac/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:15.239571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a46a66-dbc3-4037-b7a0-54af91ea2b19 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:15.249344 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a46a66-dbc3-4037-b7a0-54af91ea2b19 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:15.314688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097bf4b-1dd3-41fa-a51d-215ed39b5fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:15.315414 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097bf4b-1dd3-41fa-a51d-215ed39b5fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:15.315697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097bf4b-1dd3-41fa-a51d-215ed39b5fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:15.316434 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097bf4b-1dd3-41fa-a51d-215ed39b5fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:15.317172 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7097bf4b-1dd3-41fa-a51d-215ed39b5fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:15.318800 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7097bf4b-1dd3-41fa-a51d-215ed39b5fb5 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:15.328683 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 970/1946] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:15.332747 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de98cb-7edc-44bd-8c6a-15fd4f553bb7 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:15.343014 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de98cb-7edc-44bd-8c6a-15fd4f553bb7 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:15.348037 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a4001e-7b1b-427f-adb7-168abcdd7414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:15.348895 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4001e-7b1b-427f-adb7-168abcdd7414 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:15.349174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4001e-7b1b-427f-adb7-168abcdd7414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:15.349448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4001e-7b1b-427f-adb7-168abcdd7414 admin admin] PATH_INFO: `/v3/projects/fddd17237117482e8864620da2a460a0/users/e1ec9f2cc17b4176b65512e701eeabac/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:15.351408 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a4001e-7b1b-427f-adb7-168abcdd7414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fddd17237117482e8864620da2a460a0, user_id=e1ec9f2cc17b4176b65512e701eeabac, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:15.354020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a4001e-7b1b-427f-adb7-168abcdd7414 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:15.364485 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 977/1947] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:15 2022] PUT /identity/v3/projects/fddd17237117482e8864620da2a460a0/users/e1ec9f2cc17b4176b65512e701eeabac/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:45:15.375205 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c4620-b253-426e-ba36-a5f672fd0b34 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:15.375505 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c4620-b253-426e-ba36-a5f672fd0b34 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:15.375791 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c4620-b253-426e-ba36-a5f672fd0b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:15.403606 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d76c4620-b253-426e-ba36-a5f672fd0b34 None None] MFA Rules not processed for user `e1ec9f2cc17b4176b65512e701eeabac`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:15.404724 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d76c4620-b253-426e-ba36-a5f672fd0b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:15.452302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:15.473886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 971/1948] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:15 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:15.488580 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a4001e-7b1b-427f-adb7-168abcdd7414 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:15.490575 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-29a4001e-7b1b-427f-adb7-168abcdd7414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:15.548535 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a4001e-7b1b-427f-adb7-168abcdd7414 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:15.552402 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d469de8-b06b-45c9-9a83-81ea050f03ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:15.553042 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d469de8-b06b-45c9-9a83-81ea050f03ee service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:15.553325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d469de8-b06b-45c9-9a83-81ea050f03ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:15.553600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d469de8-b06b-45c9-9a83-81ea050f03ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:15.562352 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d469de8-b06b-45c9-9a83-81ea050f03ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:15.563812 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d469de8-b06b-45c9-9a83-81ea050f03ee service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:15.573437 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 978/1949] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:15 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:15.683453 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c4620-b253-426e-ba36-a5f672fd0b34 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:15.687545 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d76c4620-b253-426e-ba36-a5f672fd0b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:15.701951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c4620-b253-426e-ba36-a5f672fd0b34 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:15.706137 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa642466-b66a-4c2c-ad4c-26d7eda6575e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:15.706289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa642466-b66a-4c2c-ad4c-26d7eda6575e service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:15.707027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa642466-b66a-4c2c-ad4c-26d7eda6575e service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:15.707387 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa642466-b66a-4c2c-ad4c-26d7eda6575e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:15.770186 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa642466-b66a-4c2c-ad4c-26d7eda6575e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:15.771287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa642466-b66a-4c2c-ad4c-26d7eda6575e service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:15.780701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 972/1950] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:21.266322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d469de8-b06b-45c9-9a83-81ea050f03ee service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:21.372412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d469de8-b06b-45c9-9a83-81ea050f03ee service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:21.372412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f73cb0-089b-420a-a04d-e9abbee875de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:21.372412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f73cb0-089b-420a-a04d-e9abbee875de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:21.372412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f73cb0-089b-420a-a04d-e9abbee875de admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:21.373350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f73cb0-089b-420a-a04d-e9abbee875de admin admin] PATH_INFO: `/v3/users/3b89fbff0a3d4b1b970de6a8138b93bf` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:21.373350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f73cb0-089b-420a-a04d-e9abbee875de admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b89fbff0a3d4b1b970de6a8138b93bf)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:21.373350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f73cb0-089b-420a-a04d-e9abbee875de admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:21.633400 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:21.645168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d1f73cb0-089b-420a-a04d-e9abbee875de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b89fbff0a3d4b1b970de6a8138b93bf'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:21.652004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d1f73cb0-089b-420a-a04d-e9abbee875de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b89fbff0a3d4b1b970de6a8138b93bf'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:21.652657 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d1f73cb0-089b-420a-a04d-e9abbee875de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b89fbff0a3d4b1b970de6a8138b93bf'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:21.653299 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d1f73cb0-089b-420a-a04d-e9abbee875de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b89fbff0a3d4b1b970de6a8138b93bf'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:21.670087 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d1f73cb0-089b-420a-a04d-e9abbee875de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b89fbff0a3d4b1b970de6a8138b93bf'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:21.763116 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 979/1951] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:21 2022] DELETE /identity/v3/users/3b89fbff0a3d4b1b970de6a8138b93bf => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:21.768281 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f341f-cccd-4163-92a8-c5ecf92ee529 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:21.768778 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f341f-cccd-4163-92a8-c5ecf92ee529 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:21.768886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f341f-cccd-4163-92a8-c5ecf92ee529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:21.788680 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-575f341f-cccd-4163-92a8-c5ecf92ee529 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:21.789893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-575f341f-cccd-4163-92a8-c5ecf92ee529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:21.851737 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 973/1952] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:45:21.861623 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f73cb0-089b-420a-a04d-e9abbee875de admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:21.864166 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d1f73cb0-089b-420a-a04d-e9abbee875de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:21.921821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f73cb0-089b-420a-a04d-e9abbee875de admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:21.926092 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c7256-51d2-469b-beff-4c3b0f622a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:21.926770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c7256-51d2-469b-beff-4c3b0f622a3b service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:21.927052 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c7256-51d2-469b-beff-4c3b0f622a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:21.927329 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c7256-51d2-469b-beff-4c3b0f622a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:21.936804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c7256-51d2-469b-beff-4c3b0f622a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:21.938252 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c7256-51d2-469b-beff-4c3b0f622a3b service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:21.948107 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 980/1953] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:22.329715 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f341f-cccd-4163-92a8-c5ecf92ee529 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:22.331818 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-575f341f-cccd-4163-92a8-c5ecf92ee529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:22.344403 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f341f-cccd-4163-92a8-c5ecf92ee529 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:22.350506 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c6411-f695-4a8a-a884-4c6abfc5785d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:22.351393 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c6411-f695-4a8a-a884-4c6abfc5785d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:22.351674 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c6411-f695-4a8a-a884-4c6abfc5785d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:22.351943 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c6411-f695-4a8a-a884-4c6abfc5785d admin admin] PATH_INFO: `/v3/projects/d655240a40b3461f9ea914427a8a70aa` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:22.353279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c6411-f695-4a8a-a884-4c6abfc5785d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d655240a40b3461f9ea914427a8a70aa)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:22.355181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c6411-f695-4a8a-a884-4c6abfc5785d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:22.500302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ba7c6411-f695-4a8a-a884-4c6abfc5785d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd655240a40b3461f9ea914427a8a70aa', 'request_id': 'req-ba7c6411-f695-4a8a-a884-4c6abfc5785d'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:22.508610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ba7c6411-f695-4a8a-a884-4c6abfc5785d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd655240a40b3461f9ea914427a8a70aa', 'request_id': 'req-ba7c6411-f695-4a8a-a884-4c6abfc5785d'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:22.619304 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ba7c6411-f695-4a8a-a884-4c6abfc5785d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd655240a40b3461f9ea914427a8a70aa', 'request_id': 'req-ba7c6411-f695-4a8a-a884-4c6abfc5785d'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:22.631746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ba7c6411-f695-4a8a-a884-4c6abfc5785d admin admin] The token cache is being invalidate because project d655240a40b3461f9ea914427a8a70aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:22.632035 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ba7c6411-f695-4a8a-a884-4c6abfc5785d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:22.632683 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ba7c6411-f695-4a8a-a884-4c6abfc5785d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:22.634346 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 974/1954] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:22 2022] DELETE /identity/v3/projects/d655240a40b3461f9ea914427a8a70aa => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:22.638621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c7256-51d2-469b-beff-4c3b0f622a3b service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:22.650405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c7256-51d2-469b-beff-4c3b0f622a3b service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:22.709839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f67b9-bd94-4825-a9dc-dc08e4d515a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:22.710821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f67b9-bd94-4825-a9dc-dc08e4d515a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:22.711105 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f67b9-bd94-4825-a9dc-dc08e4d515a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:22.711378 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f67b9-bd94-4825-a9dc-dc08e4d515a3 admin admin] PATH_INFO: `/v3/users/e1ec9f2cc17b4176b65512e701eeabac` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:22.712583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f67b9-bd94-4825-a9dc-dc08e4d515a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1ec9f2cc17b4176b65512e701eeabac)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:22.714523 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f67b9-bd94-4825-a9dc-dc08e4d515a3 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:22.919660 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-556f67b9-bd94-4825-a9dc-dc08e4d515a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1ec9f2cc17b4176b65512e701eeabac'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:22.924887 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:22.927164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-556f67b9-bd94-4825-a9dc-dc08e4d515a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1ec9f2cc17b4176b65512e701eeabac'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:22.927845 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-556f67b9-bd94-4825-a9dc-dc08e4d515a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1ec9f2cc17b4176b65512e701eeabac'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:22.928452 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-556f67b9-bd94-4825-a9dc-dc08e4d515a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1ec9f2cc17b4176b65512e701eeabac'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:22.939709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-556f67b9-bd94-4825-a9dc-dc08e4d515a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1ec9f2cc17b4176b65512e701eeabac'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:23.047343 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 981/1955] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:22 2022] DELETE /identity/v3/users/e1ec9f2cc17b4176b65512e701eeabac => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:23.581347 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c6411-f695-4a8a-a884-4c6abfc5785d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:23.583567 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ba7c6411-f695-4a8a-a884-4c6abfc5785d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:23.597349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c6411-f695-4a8a-a884-4c6abfc5785d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:23.602639 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:23.656459 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aea247-0a07-49e1-b23e-250bc7f25f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:23.657339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aea247-0a07-49e1-b23e-250bc7f25f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:23.657626 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aea247-0a07-49e1-b23e-250bc7f25f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:23.657904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aea247-0a07-49e1-b23e-250bc7f25f07 admin admin] PATH_INFO: `/v3/projects/fddd17237117482e8864620da2a460a0` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:23.659347 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aea247-0a07-49e1-b23e-250bc7f25f07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fddd17237117482e8864620da2a460a0)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:23.661605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aea247-0a07-49e1-b23e-250bc7f25f07 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:23.790712 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-54aea247-0a07-49e1-b23e-250bc7f25f07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fddd17237117482e8864620da2a460a0', 'request_id': 'req-54aea247-0a07-49e1-b23e-250bc7f25f07'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:23.797253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-54aea247-0a07-49e1-b23e-250bc7f25f07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fddd17237117482e8864620da2a460a0', 'request_id': 'req-54aea247-0a07-49e1-b23e-250bc7f25f07'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:23.876989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-54aea247-0a07-49e1-b23e-250bc7f25f07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fddd17237117482e8864620da2a460a0', 'request_id': 'req-54aea247-0a07-49e1-b23e-250bc7f25f07'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:23.888710 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-54aea247-0a07-49e1-b23e-250bc7f25f07 admin admin] The token cache is being invalidate because project fddd17237117482e8864620da2a460a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:23.890009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-54aea247-0a07-49e1-b23e-250bc7f25f07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:23.890009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-54aea247-0a07-49e1-b23e-250bc7f25f07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:23.891511 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 975/1956] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:23 2022] DELETE /identity/v3/projects/fddd17237117482e8864620da2a460a0 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:23.896672 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfd5e7-199b-4c67-8d83-e155da413f78 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:23.897871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfd5e7-199b-4c67-8d83-e155da413f78 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:23.898154 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfd5e7-199b-4c67-8d83-e155da413f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:23.920599 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-11dfd5e7-199b-4c67-8d83-e155da413f78 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:23.920599 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-11dfd5e7-199b-4c67-8d83-e155da413f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:23.981435 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 982/1957] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:23.989704 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcd0c9-4365-48b6-b7b0-0027ee6b0af8 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:23.989989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcd0c9-4365-48b6-b7b0-0027ee6b0af8 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:23.990266 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcd0c9-4365-48b6-b7b0-0027ee6b0af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:24.009602 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3edcd0c9-4365-48b6-b7b0-0027ee6b0af8 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:24.015149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3edcd0c9-4365-48b6-b7b0-0027ee6b0af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:24.019395 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 976/1958] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:24.023722 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfd5e7-199b-4c67-8d83-e155da413f78 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:24.033360 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfd5e7-199b-4c67-8d83-e155da413f78 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:24.038507 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a878f-c0e9-471e-ab7a-7ea3976451e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:24.039184 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a878f-c0e9-471e-ab7a-7ea3976451e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:24.039466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a878f-c0e9-471e-ab7a-7ea3976451e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:24.039742 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a878f-c0e9-471e-ab7a-7ea3976451e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:24.040463 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a878f-c0e9-471e-ab7a-7ea3976451e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:24.040771 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a878f-c0e9-471e-ab7a-7ea3976451e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:24.042362 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a878f-c0e9-471e-ab7a-7ea3976451e6 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:24.055065 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 983/1959] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:24.059358 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edcd0c9-4365-48b6-b7b0-0027ee6b0af8 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:24.068938 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edcd0c9-4365-48b6-b7b0-0027ee6b0af8 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:24.073527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbe9e5-9c90-4a8a-a0c9-9600f634fa6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:24.074169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbe9e5-9c90-4a8a-a0c9-9600f634fa6a admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:24.074447 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbe9e5-9c90-4a8a-a0c9-9600f634fa6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:24.074752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbe9e5-9c90-4a8a-a0c9-9600f634fa6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:24.075433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dbe9e5-9c90-4a8a-a0c9-9600f634fa6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:24.077323 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dbe9e5-9c90-4a8a-a0c9-9600f634fa6a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:24.169604 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 977/1960] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:24 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:24.173874 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a878f-c0e9-471e-ab7a-7ea3976451e6 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:24.184305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a878f-c0e9-471e-ab7a-7ea3976451e6 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:24.240658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a5535-8270-44b1-a815-60f990ff67bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:24.241333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a5535-8270-44b1-a815-60f990ff67bd admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:24.241615 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a5535-8270-44b1-a815-60f990ff67bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:24.241888 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a5535-8270-44b1-a815-60f990ff67bd admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:24.242619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a5535-8270-44b1-a815-60f990ff67bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:24.244541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a5535-8270-44b1-a815-60f990ff67bd admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:24.334310 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 984/1961] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:24 2022] POST /identity/v3/users => generated 294 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:45:24.337220 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbe9e5-9c90-4a8a-a0c9-9600f634fa6a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:24.348628 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbe9e5-9c90-4a8a-a0c9-9600f634fa6a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:24.353482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469145e3-ab4d-4da0-9687-898290be9593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:24.354178 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469145e3-ab4d-4da0-9687-898290be9593 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:24.354466 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469145e3-ab4d-4da0-9687-898290be9593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:24.354794 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469145e3-ab4d-4da0-9687-898290be9593 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:24.355554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469145e3-ab4d-4da0-9687-898290be9593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:24.357194 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469145e3-ab4d-4da0-9687-898290be9593 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:24.369072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 978/1962] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:24.373042 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a5535-8270-44b1-a815-60f990ff67bd admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:24.383613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a5535-8270-44b1-a815-60f990ff67bd admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:24.388185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff95d30-8da6-4698-a6f4-327b25ec8830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:24.388847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff95d30-8da6-4698-a6f4-327b25ec8830 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:24.389128 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff95d30-8da6-4698-a6f4-327b25ec8830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:24.389401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff95d30-8da6-4698-a6f4-327b25ec8830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:24.390121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff95d30-8da6-4698-a6f4-327b25ec8830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:24.391745 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff95d30-8da6-4698-a6f4-327b25ec8830 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:24.402705 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 985/1963] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:24.406962 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469145e3-ab4d-4da0-9687-898290be9593 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:24.417704 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469145e3-ab4d-4da0-9687-898290be9593 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:24.422566 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd43ec-e996-4790-abf3-5462cb10771c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:24.423455 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd43ec-e996-4790-abf3-5462cb10771c admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:24.423744 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd43ec-e996-4790-abf3-5462cb10771c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:24.424023 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd43ec-e996-4790-abf3-5462cb10771c admin admin] PATH_INFO: `/v3/projects/39f73796ee4f49d59f0984d6da576759/users/2b59ac40da9f43a081443c454051c31e/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:24.439747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfd43ec-e996-4790-abf3-5462cb10771c admin admin] RBAC: Authorizing `identity:create_grant(project_id=39f73796ee4f49d59f0984d6da576759, user_id=2b59ac40da9f43a081443c454051c31e, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:24.442436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfd43ec-e996-4790-abf3-5462cb10771c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:24.511642 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 979/1964] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:24 2022] PUT /identity/v3/projects/39f73796ee4f49d59f0984d6da576759/users/2b59ac40da9f43a081443c454051c31e/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:24.523287 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4f12d-a447-48c2-91a9-229dba729db9 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:24.523583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4f12d-a447-48c2-91a9-229dba729db9 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:24.523865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4f12d-a447-48c2-91a9-229dba729db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:24.551500 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c6a4f12d-a447-48c2-91a9-229dba729db9 None None] MFA Rules not processed for user `2b59ac40da9f43a081443c454051c31e`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:24.552699 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c6a4f12d-a447-48c2-91a9-229dba729db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:24.578397 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:24.618187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 986/1965] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:24.626797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd43ec-e996-4790-abf3-5462cb10771c admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:24.667319 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:24.689547 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd43ec-e996-4790-abf3-5462cb10771c admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:24.694415 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7bd2ae-eed0-4987-bfd7-e847d0050472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:24.695101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bd2ae-eed0-4987-bfd7-e847d0050472 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:24.695380 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bd2ae-eed0-4987-bfd7-e847d0050472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:24.695651 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bd2ae-eed0-4987-bfd7-e847d0050472 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:24.696346 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7bd2ae-eed0-4987-bfd7-e847d0050472 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:24.698351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7bd2ae-eed0-4987-bfd7-e847d0050472 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:24.779645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 980/1966] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:24 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:24.783524 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4f12d-a447-48c2-91a9-229dba729db9 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:24.794997 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4f12d-a447-48c2-91a9-229dba729db9 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:24.852190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb026df-f5c1-4397-996c-7a9a20e5dd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:24.852886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb026df-f5c1-4397-996c-7a9a20e5dd77 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:24.853197 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb026df-f5c1-4397-996c-7a9a20e5dd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:24.853475 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb026df-f5c1-4397-996c-7a9a20e5dd77 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:24.854168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb026df-f5c1-4397-996c-7a9a20e5dd77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:24.856148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb026df-f5c1-4397-996c-7a9a20e5dd77 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:24.993711 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 987/1967] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:24 2022] POST /identity/v3/users => generated 299 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:24.995098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7bd2ae-eed0-4987-bfd7-e847d0050472 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:25.008362 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7bd2ae-eed0-4987-bfd7-e847d0050472 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:25.012955 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e42eb9-5c8c-4963-adc7-a1e534e16fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:25.013645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e42eb9-5c8c-4963-adc7-a1e534e16fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:25.013934 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e42eb9-5c8c-4963-adc7-a1e534e16fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:25.014210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e42eb9-5c8c-4963-adc7-a1e534e16fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:25.014975 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e42eb9-5c8c-4963-adc7-a1e534e16fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:25.016572 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e42eb9-5c8c-4963-adc7-a1e534e16fc1 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:25.031788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 981/1968] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:25.032850 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb026df-f5c1-4397-996c-7a9a20e5dd77 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:25.042855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb026df-f5c1-4397-996c-7a9a20e5dd77 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:25.047444 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7103d-3b56-4f6b-ab4c-1f5aaf861fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:25.048337 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7103d-3b56-4f6b-ab4c-1f5aaf861fe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:25.048620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7103d-3b56-4f6b-ab4c-1f5aaf861fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:25.048895 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7103d-3b56-4f6b-ab4c-1f5aaf861fe4 admin admin] PATH_INFO: `/v3/projects/e68001e9a2474de69f29cfc71b00108a/users/c20f1e2f99d04df58afa77374833524e/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:25.063980 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7103d-3b56-4f6b-ab4c-1f5aaf861fe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e68001e9a2474de69f29cfc71b00108a, user_id=c20f1e2f99d04df58afa77374833524e, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:25.067349 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7103d-3b56-4f6b-ab4c-1f5aaf861fe4 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:25.144300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 988/1969] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:25 2022] PUT /identity/v3/projects/e68001e9a2474de69f29cfc71b00108a/users/c20f1e2f99d04df58afa77374833524e/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:45:25.148169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e42eb9-5c8c-4963-adc7-a1e534e16fc1 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:25.158039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e42eb9-5c8c-4963-adc7-a1e534e16fc1 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:25.213609 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555103b-7a09-4643-b57d-4fa7bb1c0467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:25.214281 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555103b-7a09-4643-b57d-4fa7bb1c0467 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:25.214593 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555103b-7a09-4643-b57d-4fa7bb1c0467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:25.214877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555103b-7a09-4643-b57d-4fa7bb1c0467 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:25.215607 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555103b-7a09-4643-b57d-4fa7bb1c0467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:25.217222 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555103b-7a09-4643-b57d-4fa7bb1c0467 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:25.227398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 982/1970] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:25.231619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7103d-3b56-4f6b-ab4c-1f5aaf861fe4 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:25.242501 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7103d-3b56-4f6b-ab4c-1f5aaf861fe4 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:25.247052 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddea833-4a97-45c4-9e64-53c1829475c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:25.247922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddea833-4a97-45c4-9e64-53c1829475c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:25.248202 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddea833-4a97-45c4-9e64-53c1829475c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:25.248477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddea833-4a97-45c4-9e64-53c1829475c4 admin admin] PATH_INFO: `/v3/projects/e68001e9a2474de69f29cfc71b00108a/users/c20f1e2f99d04df58afa77374833524e/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:25.250408 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddea833-4a97-45c4-9e64-53c1829475c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e68001e9a2474de69f29cfc71b00108a, user_id=c20f1e2f99d04df58afa77374833524e, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:25.253712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddea833-4a97-45c4-9e64-53c1829475c4 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:25.263796 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 989/1971] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:25 2022] PUT /identity/v3/projects/e68001e9a2474de69f29cfc71b00108a/users/c20f1e2f99d04df58afa77374833524e/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:45:25.273914 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f4477-96fc-454e-97ca-8a345139c4ef None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:25.274201 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f4477-96fc-454e-97ca-8a345139c4ef None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:25.274475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f4477-96fc-454e-97ca-8a345139c4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:25.300521 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-717f4477-96fc-454e-97ca-8a345139c4ef None None] MFA Rules not processed for user `c20f1e2f99d04df58afa77374833524e`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:25.301678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-717f4477-96fc-454e-97ca-8a345139c4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:25.370805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 983/1972] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:25.382518 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddea833-4a97-45c4-9e64-53c1829475c4 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:25.384223 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-9ddea833-4a97-45c4-9e64-53c1829475c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:25.443716 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddea833-4a97-45c4-9e64-53c1829475c4 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:25.447376 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec18fa-45df-429a-8bee-213feb70bdec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:25.448028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec18fa-45df-429a-8bee-213feb70bdec service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:25.448312 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec18fa-45df-429a-8bee-213feb70bdec service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:25.448590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec18fa-45df-429a-8bee-213feb70bdec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:25.458751 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec18fa-45df-429a-8bee-213feb70bdec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:25.460515 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec18fa-45df-429a-8bee-213feb70bdec service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:25.470361 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 990/1973] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:25.566419 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f4477-96fc-454e-97ca-8a345139c4ef None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:25.602437 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:25.632894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f4477-96fc-454e-97ca-8a345139c4ef None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:25.637761 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05c255-d226-4739-a97e-bf3a1002cece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:25.638672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05c255-d226-4739-a97e-bf3a1002cece admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:25.638978 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05c255-d226-4739-a97e-bf3a1002cece admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:25.639258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05c255-d226-4739-a97e-bf3a1002cece admin admin] PATH_INFO: `/v3/users/2b59ac40da9f43a081443c454051c31e` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:25.640482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05c255-d226-4739-a97e-bf3a1002cece admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b59ac40da9f43a081443c454051c31e)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:25.642460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05c255-d226-4739-a97e-bf3a1002cece admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:25.821773 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-da05c255-d226-4739-a97e-bf3a1002cece admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b59ac40da9f43a081443c454051c31e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:25.828867 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-da05c255-d226-4739-a97e-bf3a1002cece admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b59ac40da9f43a081443c454051c31e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:25.829552 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-da05c255-d226-4739-a97e-bf3a1002cece admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b59ac40da9f43a081443c454051c31e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:25.830182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-da05c255-d226-4739-a97e-bf3a1002cece admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b59ac40da9f43a081443c454051c31e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:25.842930 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-da05c255-d226-4739-a97e-bf3a1002cece admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b59ac40da9f43a081443c454051c31e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:25.939313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 984/1974] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:25 2022] DELETE /identity/v3/users/2b59ac40da9f43a081443c454051c31e => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:25.944481 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9515048-d889-4ccf-b389-9902965f79a0 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:25.944791 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9515048-d889-4ccf-b389-9902965f79a0 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:25.945067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9515048-d889-4ccf-b389-9902965f79a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:25.963112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-b9515048-d889-4ccf-b389-9902965f79a0 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:25.964241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b9515048-d889-4ccf-b389-9902965f79a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:25.981954 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:26.026183 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 991/1975] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:26.036061 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05c255-d226-4739-a97e-bf3a1002cece admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:26.037504 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-da05c255-d226-4739-a97e-bf3a1002cece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:26.095657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05c255-d226-4739-a97e-bf3a1002cece admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:26.099485 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83bb2c-2cb1-4fa9-8a65-598c584c371b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:26.100136 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83bb2c-2cb1-4fa9-8a65-598c584c371b service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:26.100417 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83bb2c-2cb1-4fa9-8a65-598c584c371b service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:26.100689 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83bb2c-2cb1-4fa9-8a65-598c584c371b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:26.118577 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83bb2c-2cb1-4fa9-8a65-598c584c371b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:26.118577 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83bb2c-2cb1-4fa9-8a65-598c584c371b service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:26.123841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 985/1976] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:26.522953 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9515048-d889-4ccf-b389-9902965f79a0 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:26.524359 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b9515048-d889-4ccf-b389-9902965f79a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:26.538304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9515048-d889-4ccf-b389-9902965f79a0 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:26.543036 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2d8b4-2203-4bd4-a958-3809a51521a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:26.543853 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2d8b4-2203-4bd4-a958-3809a51521a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:26.544167 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2d8b4-2203-4bd4-a958-3809a51521a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:26.544441 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2d8b4-2203-4bd4-a958-3809a51521a3 admin admin] PATH_INFO: `/v3/projects/39f73796ee4f49d59f0984d6da576759` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:26.545625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2d8b4-2203-4bd4-a958-3809a51521a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39f73796ee4f49d59f0984d6da576759)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:26.548315 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2d8b4-2203-4bd4-a958-3809a51521a3 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:26.697359 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-89d2d8b4-2203-4bd4-a958-3809a51521a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39f73796ee4f49d59f0984d6da576759', 'request_id': 'req-89d2d8b4-2203-4bd4-a958-3809a51521a3'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:26.707793 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-89d2d8b4-2203-4bd4-a958-3809a51521a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39f73796ee4f49d59f0984d6da576759', 'request_id': 'req-89d2d8b4-2203-4bd4-a958-3809a51521a3'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:26.785358 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-89d2d8b4-2203-4bd4-a958-3809a51521a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39f73796ee4f49d59f0984d6da576759', 'request_id': 'req-89d2d8b4-2203-4bd4-a958-3809a51521a3'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:26.796516 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-89d2d8b4-2203-4bd4-a958-3809a51521a3 admin admin] The token cache is being invalidate because project 39f73796ee4f49d59f0984d6da576759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:26.796802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-89d2d8b4-2203-4bd4-a958-3809a51521a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:26.797835 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-89d2d8b4-2203-4bd4-a958-3809a51521a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:26.799438 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 992/1977] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:26 2022] DELETE /identity/v3/projects/39f73796ee4f49d59f0984d6da576759 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:26.804716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83bb2c-2cb1-4fa9-8a65-598c584c371b service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:26.814674 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83bb2c-2cb1-4fa9-8a65-598c584c371b service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:26.863868 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:26.872216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce6456-d765-456c-8da1-d9456f7b5c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:26.873082 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce6456-d765-456c-8da1-d9456f7b5c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:26.877475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce6456-d765-456c-8da1-d9456f7b5c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:26.877475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce6456-d765-456c-8da1-d9456f7b5c5b admin admin] PATH_INFO: `/v3/users/c20f1e2f99d04df58afa77374833524e` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:26.877475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce6456-d765-456c-8da1-d9456f7b5c5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c20f1e2f99d04df58afa77374833524e)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:26.877475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce6456-d765-456c-8da1-d9456f7b5c5b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:27.094128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4cce6456-d765-456c-8da1-d9456f7b5c5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c20f1e2f99d04df58afa77374833524e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:27.101821 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4cce6456-d765-456c-8da1-d9456f7b5c5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c20f1e2f99d04df58afa77374833524e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:27.102468 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4cce6456-d765-456c-8da1-d9456f7b5c5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c20f1e2f99d04df58afa77374833524e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:27.103101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4cce6456-d765-456c-8da1-d9456f7b5c5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c20f1e2f99d04df58afa77374833524e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:27.115908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4cce6456-d765-456c-8da1-d9456f7b5c5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c20f1e2f99d04df58afa77374833524e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:27.207337 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 986/1978] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:26 2022] DELETE /identity/v3/users/c20f1e2f99d04df58afa77374833524e => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:27.615051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2d8b4-2203-4bd4-a958-3809a51521a3 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:27.616593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-89d2d8b4-2203-4bd4-a958-3809a51521a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:27.629776 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2d8b4-2203-4bd4-a958-3809a51521a3 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:27.689588 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e332ef4-147d-4e4a-8fc6-e484628fb17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:27.690463 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e332ef4-147d-4e4a-8fc6-e484628fb17b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:27.690841 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e332ef4-147d-4e4a-8fc6-e484628fb17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:27.691118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e332ef4-147d-4e4a-8fc6-e484628fb17b admin admin] PATH_INFO: `/v3/projects/e68001e9a2474de69f29cfc71b00108a` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:27.692309 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e332ef4-147d-4e4a-8fc6-e484628fb17b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e68001e9a2474de69f29cfc71b00108a)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:27.694200 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e332ef4-147d-4e4a-8fc6-e484628fb17b admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:27.849793 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:27.851026 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-1e332ef4-147d-4e4a-8fc6-e484628fb17b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e68001e9a2474de69f29cfc71b00108a', 'request_id': 'req-1e332ef4-147d-4e4a-8fc6-e484628fb17b'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:27.857217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-1e332ef4-147d-4e4a-8fc6-e484628fb17b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e68001e9a2474de69f29cfc71b00108a', 'request_id': 'req-1e332ef4-147d-4e4a-8fc6-e484628fb17b'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:27.962342 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-1e332ef4-147d-4e4a-8fc6-e484628fb17b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e68001e9a2474de69f29cfc71b00108a', 'request_id': 'req-1e332ef4-147d-4e4a-8fc6-e484628fb17b'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:27.973113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-1e332ef4-147d-4e4a-8fc6-e484628fb17b admin admin] The token cache is being invalidate because project e68001e9a2474de69f29cfc71b00108a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:27.978720 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-1e332ef4-147d-4e4a-8fc6-e484628fb17b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:27.978720 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-1e332ef4-147d-4e4a-8fc6-e484628fb17b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:27.978720 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 993/1979] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:27 2022] DELETE /identity/v3/projects/e68001e9a2474de69f29cfc71b00108a => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:27.980517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb58890-84b5-4e77-97ed-6606b8804033 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:27.980969 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb58890-84b5-4e77-97ed-6606b8804033 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:27.981085 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb58890-84b5-4e77-97ed-6606b8804033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:27.999013 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-bbb58890-84b5-4e77-97ed-6606b8804033 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:28.000276 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bbb58890-84b5-4e77-97ed-6606b8804033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:28.059950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 987/1980] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:45:28.068824 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f273701-1e15-4a7a-a5d6-e41ea4176022 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:28.069113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f273701-1e15-4a7a-a5d6-e41ea4176022 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:28.069393 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f273701-1e15-4a7a-a5d6-e41ea4176022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:28.088156 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-2f273701-1e15-4a7a-a5d6-e41ea4176022 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:28.089349 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-2f273701-1e15-4a7a-a5d6-e41ea4176022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:28.097215 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 994/1981] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:28.101519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb58890-84b5-4e77-97ed-6606b8804033 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:28.111002 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb58890-84b5-4e77-97ed-6606b8804033 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:28.116567 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077530e-fd6b-4a3a-892b-0e37c3d47e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:28.117212 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077530e-fd6b-4a3a-892b-0e37c3d47e95 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:28.117492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077530e-fd6b-4a3a-892b-0e37c3d47e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:28.117763 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077530e-fd6b-4a3a-892b-0e37c3d47e95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:28.118479 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077530e-fd6b-4a3a-892b-0e37c3d47e95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:28.118816 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077530e-fd6b-4a3a-892b-0e37c3d47e95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:28.120417 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077530e-fd6b-4a3a-892b-0e37c3d47e95 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:28.132934 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 988/1982] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:28.137362 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f273701-1e15-4a7a-a5d6-e41ea4176022 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:28.146917 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f273701-1e15-4a7a-a5d6-e41ea4176022 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:28.151640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8b130-f2b8-4247-9b28-5737227d78fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:28.152292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8b130-f2b8-4247-9b28-5737227d78fd admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:28.152572 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8b130-f2b8-4247-9b28-5737227d78fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:28.152845 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8b130-f2b8-4247-9b28-5737227d78fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:28.153534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8b130-f2b8-4247-9b28-5737227d78fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:28.155720 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8b130-f2b8-4247-9b28-5737227d78fd admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:28.276930 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 995/1983] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:28 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:28.281161 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077530e-fd6b-4a3a-892b-0e37c3d47e95 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:28.292297 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077530e-fd6b-4a3a-892b-0e37c3d47e95 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:28.349968 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d011e6-3864-430c-9749-dad0d6ddaaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:28.350688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d011e6-3864-430c-9749-dad0d6ddaaca admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:28.350976 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d011e6-3864-430c-9749-dad0d6ddaaca admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:28.351252 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d011e6-3864-430c-9749-dad0d6ddaaca admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:28.352037 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d011e6-3864-430c-9749-dad0d6ddaaca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:28.353967 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d011e6-3864-430c-9749-dad0d6ddaaca admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:28.360708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:28.480932 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 989/1984] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:28 2022] POST /identity/v3/users => generated 290 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:45:28.493526 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8b130-f2b8-4247-9b28-5737227d78fd admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:28.500393 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8b130-f2b8-4247-9b28-5737227d78fd admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:28.505438 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f2627-2e21-4aec-afa3-7926718217e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:28.506145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f2627-2e21-4aec-afa3-7926718217e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:28.506479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f2627-2e21-4aec-afa3-7926718217e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:28.506810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f2627-2e21-4aec-afa3-7926718217e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:28.507541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f2627-2e21-4aec-afa3-7926718217e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:28.509144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f2627-2e21-4aec-afa3-7926718217e8 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:28.523774 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 996/1985] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:28.524705 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d011e6-3864-430c-9749-dad0d6ddaaca admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:28.535887 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d011e6-3864-430c-9749-dad0d6ddaaca admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:28.541148 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7304eb-8c9c-4e1c-bd19-c33663159169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:28.541820 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7304eb-8c9c-4e1c-bd19-c33663159169 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:28.542105 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7304eb-8c9c-4e1c-bd19-c33663159169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:28.542382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7304eb-8c9c-4e1c-bd19-c33663159169 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:28.543143 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7304eb-8c9c-4e1c-bd19-c33663159169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:28.544838 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7304eb-8c9c-4e1c-bd19-c33663159169 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:28.556530 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 990/1986] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:45:28.560662 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f2627-2e21-4aec-afa3-7926718217e8 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:28.570619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f2627-2e21-4aec-afa3-7926718217e8 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:28.575117 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3d144-5e47-4684-b8ea-0b87bebfcb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:28.575989 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3d144-5e47-4684-b8ea-0b87bebfcb6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:28.576269 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3d144-5e47-4684-b8ea-0b87bebfcb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:28.576539 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3d144-5e47-4684-b8ea-0b87bebfcb6b admin admin] PATH_INFO: `/v3/projects/2b6fdd477c8c496eb2b92b9abe4bdaf2/users/1bb46c4128a9484dbde0f5dc051e22da/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:28.591842 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3d144-5e47-4684-b8ea-0b87bebfcb6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b6fdd477c8c496eb2b92b9abe4bdaf2, user_id=1bb46c4128a9484dbde0f5dc051e22da, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:28.603446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3d144-5e47-4684-b8ea-0b87bebfcb6b admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:28.700888 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 997/1987] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:28 2022] PUT /identity/v3/projects/2b6fdd477c8c496eb2b92b9abe4bdaf2/users/1bb46c4128a9484dbde0f5dc051e22da/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:28.712028 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2b5de-6ea7-4588-8281-8df2e77f5b5a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:28.712314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2b5de-6ea7-4588-8281-8df2e77f5b5a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:28.712657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2b5de-6ea7-4588-8281-8df2e77f5b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:28.797754 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a8c2b5de-6ea7-4588-8281-8df2e77f5b5a None None] MFA Rules not processed for user `1bb46c4128a9484dbde0f5dc051e22da`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:28.798987 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a8c2b5de-6ea7-4588-8281-8df2e77f5b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:28.865216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 991/1988] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:28.869817 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3d144-5e47-4684-b8ea-0b87bebfcb6b admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:28.933165 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3d144-5e47-4684-b8ea-0b87bebfcb6b admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:28.938048 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f5a7e-36fd-41a2-824c-57f39897447a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:28.938739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f5a7e-36fd-41a2-824c-57f39897447a admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:28.939024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f5a7e-36fd-41a2-824c-57f39897447a admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:28.939300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f5a7e-36fd-41a2-824c-57f39897447a admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:28.940035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f5a7e-36fd-41a2-824c-57f39897447a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:28.942221 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f5a7e-36fd-41a2-824c-57f39897447a admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:29.039096 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 998/1989] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:28 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:29.043100 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2b5de-6ea7-4588-8281-8df2e77f5b5a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:29.054477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2b5de-6ea7-4588-8281-8df2e77f5b5a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:29.111066 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4a03d-fe8a-4454-a3ef-dc02259914a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:29.111738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4a03d-fe8a-4454-a3ef-dc02259914a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:29.112026 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4a03d-fe8a-4454-a3ef-dc02259914a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:29.112308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4a03d-fe8a-4454-a3ef-dc02259914a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:29.113027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4a03d-fe8a-4454-a3ef-dc02259914a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:29.115129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4a03d-fe8a-4454-a3ef-dc02259914a6 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:29.214521 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 992/1990] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:29 2022] POST /identity/v3/users => generated 296 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:29.218751 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f5a7e-36fd-41a2-824c-57f39897447a admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:29.229582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f5a7e-36fd-41a2-824c-57f39897447a admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:29.234120 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9a429-5a96-43f2-b2f8-c400897b63cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:29.234801 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9a429-5a96-43f2-b2f8-c400897b63cd admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:29.235135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9a429-5a96-43f2-b2f8-c400897b63cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:29.235363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9a429-5a96-43f2-b2f8-c400897b63cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:29.236081 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd9a429-5a96-43f2-b2f8-c400897b63cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:29.237677 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd9a429-5a96-43f2-b2f8-c400897b63cd admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:29.248289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 999/1991] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:29.252309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4a03d-fe8a-4454-a3ef-dc02259914a6 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:29.262677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4a03d-fe8a-4454-a3ef-dc02259914a6 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:29.267535 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb769d5-c4a3-4eba-89f5-abda146c8c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:29.268412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb769d5-c4a3-4eba-89f5-abda146c8c1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:29.268698 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb769d5-c4a3-4eba-89f5-abda146c8c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:29.268971 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb769d5-c4a3-4eba-89f5-abda146c8c1e admin admin] PATH_INFO: `/v3/projects/6ec4e2ad1298456190c84051430500c1/users/b1ab09632cef46349055f8ebe5a1c38d/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:29.286092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb769d5-c4a3-4eba-89f5-abda146c8c1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ec4e2ad1298456190c84051430500c1, user_id=b1ab09632cef46349055f8ebe5a1c38d, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:29.288754 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb769d5-c4a3-4eba-89f5-abda146c8c1e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:29.395529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 993/1992] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:29 2022] PUT /identity/v3/projects/6ec4e2ad1298456190c84051430500c1/users/b1ab09632cef46349055f8ebe5a1c38d/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:29.399503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9a429-5a96-43f2-b2f8-c400897b63cd admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:29.409596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9a429-5a96-43f2-b2f8-c400897b63cd admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:29.455816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:29.471175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1c395-af32-4f68-8676-aebde8a31b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:29.471972 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1c395-af32-4f68-8676-aebde8a31b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:29.472269 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1c395-af32-4f68-8676-aebde8a31b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:29.472573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1c395-af32-4f68-8676-aebde8a31b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:29.473348 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1c395-af32-4f68-8676-aebde8a31b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:29.474984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1c395-af32-4f68-8676-aebde8a31b43 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:29.498925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1000/1993] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:29.500143 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb769d5-c4a3-4eba-89f5-abda146c8c1e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:29.512067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb769d5-c4a3-4eba-89f5-abda146c8c1e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:29.516907 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29bb42-a808-4319-ab92-2e12547eda0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:29.517826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29bb42-a808-4319-ab92-2e12547eda0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:29.518106 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29bb42-a808-4319-ab92-2e12547eda0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:29.518383 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29bb42-a808-4319-ab92-2e12547eda0d admin admin] PATH_INFO: `/v3/projects/6ec4e2ad1298456190c84051430500c1/users/b1ab09632cef46349055f8ebe5a1c38d/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:29.522458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29bb42-a808-4319-ab92-2e12547eda0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ec4e2ad1298456190c84051430500c1, user_id=b1ab09632cef46349055f8ebe5a1c38d, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:29.525142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29bb42-a808-4319-ab92-2e12547eda0d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:29.536328 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 994/1994] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:29 2022] PUT /identity/v3/projects/6ec4e2ad1298456190c84051430500c1/users/b1ab09632cef46349055f8ebe5a1c38d/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:29.546392 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505547d1-330f-451d-abd5-a833ceb239d4 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:29.546808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505547d1-330f-451d-abd5-a833ceb239d4 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:29.547159 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505547d1-330f-451d-abd5-a833ceb239d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:29.574816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-505547d1-330f-451d-abd5-a833ceb239d4 None None] MFA Rules not processed for user `b1ab09632cef46349055f8ebe5a1c38d`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:29.576038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-505547d1-330f-451d-abd5-a833ceb239d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:29.664408 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1001/1995] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:29 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:29.684156 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29bb42-a808-4319-ab92-2e12547eda0d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:29.695337 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-aa29bb42-a808-4319-ab92-2e12547eda0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:29.725779 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:29.769902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29bb42-a808-4319-ab92-2e12547eda0d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:29.776335 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df78e7-d242-4d8b-aec0-e504ee267bb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:29.777116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df78e7-d242-4d8b-aec0-e504ee267bb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:29.777519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df78e7-d242-4d8b-aec0-e504ee267bb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:29.777890 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df78e7-d242-4d8b-aec0-e504ee267bb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:29.790329 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df78e7-d242-4d8b-aec0-e504ee267bb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:29.791878 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df78e7-d242-4d8b-aec0-e504ee267bb4 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:29.804476 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 995/1996] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:29 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:31.073071 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505547d1-330f-451d-abd5-a833ceb239d4 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:31.083064 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505547d1-330f-451d-abd5-a833ceb239d4 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:31.088082 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603aa50-c1c9-4103-9d7e-5b57358c7b5c tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ab09632cef46349055f8ebe5a1c38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ec4e2ad1298456190c84051430500c1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:31.092333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603aa50-c1c9-4103-9d7e-5b57358c7b5c tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:31.092333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603aa50-c1c9-4103-9d7e-5b57358c7b5c tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:31.092333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603aa50-c1c9-4103-9d7e-5b57358c7b5c tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:31.092333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603aa50-c1c9-4103-9d7e-5b57358c7b5c tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:31.092333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603aa50-c1c9-4103-9d7e-5b57358c7b5c tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:31.092759 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603aa50-c1c9-4103-9d7e-5b57358c7b5c tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:31.103667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1002/1997] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:31.108079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df78e7-d242-4d8b-aec0-e504ee267bb4 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:31.117984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df78e7-d242-4d8b-aec0-e504ee267bb4 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:31.122608 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc35303-eebc-4277-b8ae-5fb8144350f3 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ab09632cef46349055f8ebe5a1c38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ec4e2ad1298456190c84051430500c1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:31.123289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc35303-eebc-4277-b8ae-5fb8144350f3 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:31.123577 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc35303-eebc-4277-b8ae-5fb8144350f3 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:31.123882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc35303-eebc-4277-b8ae-5fb8144350f3 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:31.124583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc35303-eebc-4277-b8ae-5fb8144350f3 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:31.126507 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc35303-eebc-4277-b8ae-5fb8144350f3 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:31.271449 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 996/1998] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:31 2022] POST /identity/v3/projects => generated 355 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:31.272594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603aa50-c1c9-4103-9d7e-5b57358c7b5c tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:31.283346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603aa50-c1c9-4103-9d7e-5b57358c7b5c tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:31.346933 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8dbe0-2266-406e-8b45-99565567f791 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ab09632cef46349055f8ebe5a1c38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ec4e2ad1298456190c84051430500c1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:31.347868 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8dbe0-2266-406e-8b45-99565567f791 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:31.348480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8dbe0-2266-406e-8b45-99565567f791 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:31.348566 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8dbe0-2266-406e-8b45-99565567f791 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:31.349321 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8dbe0-2266-406e-8b45-99565567f791 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:31.349694 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8dbe0-2266-406e-8b45-99565567f791 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:31.351345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8dbe0-2266-406e-8b45-99565567f791 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:31.365470 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1003/1999] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:31.444230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc35303-eebc-4277-b8ae-5fb8144350f3 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:31.462381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc35303-eebc-4277-b8ae-5fb8144350f3 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:31.472076 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc23919-580a-42a5-8849-4384a090ccb9 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1ab09632cef46349055f8ebe5a1c38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ec4e2ad1298456190c84051430500c1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:31.473805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc23919-580a-42a5-8849-4384a090ccb9 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:31.474229 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc23919-580a-42a5-8849-4384a090ccb9 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:31.475185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc23919-580a-42a5-8849-4384a090ccb9 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] PATH_INFO: `/v3/projects/637f3f9998ed4cbaad9063cc974920e6` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:31.476939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc23919-580a-42a5-8849-4384a090ccb9 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] RBAC: Authorizing `identity:delete_project(project_id=637f3f9998ed4cbaad9063cc974920e6)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:31.491276 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc23919-580a-42a5-8849-4384a090ccb9 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:31.660605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3fc23919-580a-42a5-8849-4384a090ccb9 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '637f3f9998ed4cbaad9063cc974920e6', 'request_id': 'req-3fc23919-580a-42a5-8849-4384a090ccb9'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:31.663853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3fc23919-580a-42a5-8849-4384a090ccb9 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '637f3f9998ed4cbaad9063cc974920e6', 'request_id': 'req-3fc23919-580a-42a5-8849-4384a090ccb9'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:31.728112 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3fc23919-580a-42a5-8849-4384a090ccb9 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '637f3f9998ed4cbaad9063cc974920e6', 'request_id': 'req-3fc23919-580a-42a5-8849-4384a090ccb9'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:31.739884 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3fc23919-580a-42a5-8849-4384a090ccb9 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] The token cache is being invalidate because project 637f3f9998ed4cbaad9063cc974920e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:31.740170 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3fc23919-580a-42a5-8849-4384a090ccb9 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:31.740826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3fc23919-580a-42a5-8849-4384a090ccb9 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:31.742395 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 997/2000] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:31 2022] DELETE /identity/v3/projects/637f3f9998ed4cbaad9063cc974920e6 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:32.370826 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8dbe0-2266-406e-8b45-99565567f791 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:32.372638 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-84c8dbe0-2266-406e-8b45-99565567f791 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:32.427324 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:32.443083 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8dbe0-2266-406e-8b45-99565567f791 tempest-VolumeQuotaClassesTest-414589632 tempest-VolumeQuotaClassesTest-414589632-project-admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:32.507856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2af67-1b78-474d-a7bf-522cc8899ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:32.508846 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2af67-1b78-474d-a7bf-522cc8899ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:32.509129 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2af67-1b78-474d-a7bf-522cc8899ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:32.509589 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2af67-1b78-474d-a7bf-522cc8899ec1 admin admin] PATH_INFO: `/v3/users/1bb46c4128a9484dbde0f5dc051e22da` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:32.510696 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2af67-1b78-474d-a7bf-522cc8899ec1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bb46c4128a9484dbde0f5dc051e22da)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:32.512653 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2af67-1b78-474d-a7bf-522cc8899ec1 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:32.702792 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-8ca2af67-1b78-474d-a7bf-522cc8899ec1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bb46c4128a9484dbde0f5dc051e22da'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:32.710111 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-8ca2af67-1b78-474d-a7bf-522cc8899ec1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bb46c4128a9484dbde0f5dc051e22da'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:32.710806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-8ca2af67-1b78-474d-a7bf-522cc8899ec1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bb46c4128a9484dbde0f5dc051e22da'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:32.711412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-8ca2af67-1b78-474d-a7bf-522cc8899ec1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bb46c4128a9484dbde0f5dc051e22da'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:32.723188 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-8ca2af67-1b78-474d-a7bf-522cc8899ec1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bb46c4128a9484dbde0f5dc051e22da'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:32.822718 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1004/2001] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:32 2022] DELETE /identity/v3/users/1bb46c4128a9484dbde0f5dc051e22da => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:45:32.827214 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6bf97-7d20-47b2-b47f-be8519f84b24 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:32.829445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6bf97-7d20-47b2-b47f-be8519f84b24 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:32.830034 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6bf97-7d20-47b2-b47f-be8519f84b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:32.851844 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-13e6bf97-7d20-47b2-b47f-be8519f84b24 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:32.853210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-13e6bf97-7d20-47b2-b47f-be8519f84b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:32.924999 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 998/2002] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:45:32.935666 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2af67-1b78-474d-a7bf-522cc8899ec1 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:32.937106 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-8ca2af67-1b78-474d-a7bf-522cc8899ec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:33.006418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2af67-1b78-474d-a7bf-522cc8899ec1 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:33.010339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa4c2a-51f2-40cb-83ce-9564912bc6d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:33.011066 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa4c2a-51f2-40cb-83ce-9564912bc6d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:33.011867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa4c2a-51f2-40cb-83ce-9564912bc6d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:33.011867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa4c2a-51f2-40cb-83ce-9564912bc6d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:33.022961 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa4c2a-51f2-40cb-83ce-9564912bc6d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:33.024903 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa4c2a-51f2-40cb-83ce-9564912bc6d1 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:33.040297 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1005/2003] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:34.304068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6bf97-7d20-47b2-b47f-be8519f84b24 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:34.305590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-13e6bf97-7d20-47b2-b47f-be8519f84b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:34.318439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6bf97-7d20-47b2-b47f-be8519f84b24 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:34.323240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080effa7-6aa9-4cde-839e-6c725a1261fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:34.324177 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080effa7-6aa9-4cde-839e-6c725a1261fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:34.324539 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080effa7-6aa9-4cde-839e-6c725a1261fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:34.324897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080effa7-6aa9-4cde-839e-6c725a1261fe admin admin] PATH_INFO: `/v3/projects/2b6fdd477c8c496eb2b92b9abe4bdaf2` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:34.326167 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080effa7-6aa9-4cde-839e-6c725a1261fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b6fdd477c8c496eb2b92b9abe4bdaf2)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:34.328180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080effa7-6aa9-4cde-839e-6c725a1261fe admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:34.350591 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:34.448963 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-080effa7-6aa9-4cde-839e-6c725a1261fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b6fdd477c8c496eb2b92b9abe4bdaf2', 'request_id': 'req-080effa7-6aa9-4cde-839e-6c725a1261fe'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:34.455541 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-080effa7-6aa9-4cde-839e-6c725a1261fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b6fdd477c8c496eb2b92b9abe4bdaf2', 'request_id': 'req-080effa7-6aa9-4cde-839e-6c725a1261fe'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:34.499741 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-080effa7-6aa9-4cde-839e-6c725a1261fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b6fdd477c8c496eb2b92b9abe4bdaf2', 'request_id': 'req-080effa7-6aa9-4cde-839e-6c725a1261fe'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:34.510733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-080effa7-6aa9-4cde-839e-6c725a1261fe admin admin] The token cache is being invalidate because project 2b6fdd477c8c496eb2b92b9abe4bdaf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:34.511030 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-080effa7-6aa9-4cde-839e-6c725a1261fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:34.512286 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-080effa7-6aa9-4cde-839e-6c725a1261fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:34.513641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 999/2004] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:34 2022] DELETE /identity/v3/projects/2b6fdd477c8c496eb2b92b9abe4bdaf2 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:34.517327 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa4c2a-51f2-40cb-83ce-9564912bc6d1 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:34.527862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa4c2a-51f2-40cb-83ce-9564912bc6d1 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:34.542695 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:34.592939 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76911e1f-9b0b-4883-b67f-f1ba40bae8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:34.593855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76911e1f-9b0b-4883-b67f-f1ba40bae8b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:34.594137 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76911e1f-9b0b-4883-b67f-f1ba40bae8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:34.594462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76911e1f-9b0b-4883-b67f-f1ba40bae8b5 admin admin] PATH_INFO: `/v3/users/b1ab09632cef46349055f8ebe5a1c38d` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:34.595754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76911e1f-9b0b-4883-b67f-f1ba40bae8b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1ab09632cef46349055f8ebe5a1c38d)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:34.598675 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76911e1f-9b0b-4883-b67f-f1ba40bae8b5 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:34.813498 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-76911e1f-9b0b-4883-b67f-f1ba40bae8b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1ab09632cef46349055f8ebe5a1c38d'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:34.818554 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-76911e1f-9b0b-4883-b67f-f1ba40bae8b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1ab09632cef46349055f8ebe5a1c38d'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:34.819254 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-76911e1f-9b0b-4883-b67f-f1ba40bae8b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1ab09632cef46349055f8ebe5a1c38d'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:34.819866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-76911e1f-9b0b-4883-b67f-f1ba40bae8b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1ab09632cef46349055f8ebe5a1c38d'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:34.832541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-76911e1f-9b0b-4883-b67f-f1ba40bae8b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1ab09632cef46349055f8ebe5a1c38d'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:34.923518 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1006/2005] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:34 2022] DELETE /identity/v3/users/b1ab09632cef46349055f8ebe5a1c38d => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:35.404330 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080effa7-6aa9-4cde-839e-6c725a1261fe admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:35.404330 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-080effa7-6aa9-4cde-839e-6c725a1261fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:35.415116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080effa7-6aa9-4cde-839e-6c725a1261fe admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:35.500730 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac7621-96d7-4924-b32d-2b65c2fe8f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:35.501952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac7621-96d7-4924-b32d-2b65c2fe8f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:35.502335 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac7621-96d7-4924-b32d-2b65c2fe8f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:35.502744 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac7621-96d7-4924-b32d-2b65c2fe8f1b admin admin] PATH_INFO: `/v3/projects/6ec4e2ad1298456190c84051430500c1` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:35.505235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac7621-96d7-4924-b32d-2b65c2fe8f1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ec4e2ad1298456190c84051430500c1)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:35.507863 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac7621-96d7-4924-b32d-2b65c2fe8f1b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:35.633734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b8ac7621-96d7-4924-b32d-2b65c2fe8f1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ec4e2ad1298456190c84051430500c1', 'request_id': 'req-b8ac7621-96d7-4924-b32d-2b65c2fe8f1b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:35.641449 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b8ac7621-96d7-4924-b32d-2b65c2fe8f1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ec4e2ad1298456190c84051430500c1', 'request_id': 'req-b8ac7621-96d7-4924-b32d-2b65c2fe8f1b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:35.699424 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b8ac7621-96d7-4924-b32d-2b65c2fe8f1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ec4e2ad1298456190c84051430500c1', 'request_id': 'req-b8ac7621-96d7-4924-b32d-2b65c2fe8f1b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:35.738635 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b8ac7621-96d7-4924-b32d-2b65c2fe8f1b admin admin] The token cache is being invalidate because project 6ec4e2ad1298456190c84051430500c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:35.738936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b8ac7621-96d7-4924-b32d-2b65c2fe8f1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:35.741801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b8ac7621-96d7-4924-b32d-2b65c2fe8f1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:35.744027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1000/2006] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:35 2022] DELETE /identity/v3/projects/6ec4e2ad1298456190c84051430500c1 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:35.749501 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98001077-bfd4-4bad-b5a6-a0e2a3bb4ea3 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:35.749977 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98001077-bfd4-4bad-b5a6-a0e2a3bb4ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:35.750078 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98001077-bfd4-4bad-b5a6-a0e2a3bb4ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:35.783431 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-98001077-bfd4-4bad-b5a6-a0e2a3bb4ea3 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:35.784740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-98001077-bfd4-4bad-b5a6-a0e2a3bb4ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:35.868162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1007/2007] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:35.875867 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc19c4-9085-4819-b8ea-e7bb6c745b85 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:35.876170 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc19c4-9085-4819-b8ea-e7bb6c745b85 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:35.876484 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc19c4-9085-4819-b8ea-e7bb6c745b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:35.897978 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-80dc19c4-9085-4819-b8ea-e7bb6c745b85 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:35.899327 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-80dc19c4-9085-4819-b8ea-e7bb6c745b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:35.908043 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1001/2008] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:35.912801 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98001077-bfd4-4bad-b5a6-a0e2a3bb4ea3 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:35.922439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98001077-bfd4-4bad-b5a6-a0e2a3bb4ea3 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:35.927425 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20862eda-1c5a-443b-b70c-9369a54fe4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:35.928173 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20862eda-1c5a-443b-b70c-9369a54fe4d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:35.928527 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20862eda-1c5a-443b-b70c-9369a54fe4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:35.928874 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20862eda-1c5a-443b-b70c-9369a54fe4d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:35.929688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20862eda-1c5a-443b-b70c-9369a54fe4d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:35.930084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20862eda-1c5a-443b-b70c-9369a54fe4d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:35.931808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20862eda-1c5a-443b-b70c-9369a54fe4d2 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:35.944158 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1008/2009] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:35.948793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc19c4-9085-4819-b8ea-e7bb6c745b85 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:35.958501 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc19c4-9085-4819-b8ea-e7bb6c745b85 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:35.963842 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746d052-9115-4111-8252-206043936d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:35.964579 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746d052-9115-4111-8252-206043936d36 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:35.964870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746d052-9115-4111-8252-206043936d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:35.965147 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746d052-9115-4111-8252-206043936d36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:35.965861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746d052-9115-4111-8252-206043936d36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:35.967819 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746d052-9115-4111-8252-206043936d36 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:36.105893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1002/2010] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:35 2022] POST /identity/v3/projects => generated 389 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:36.109952 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20862eda-1c5a-443b-b70c-9369a54fe4d2 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:36.120661 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20862eda-1c5a-443b-b70c-9369a54fe4d2 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:36.145183 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:36.185955 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127c6c7-a855-4c64-8e00-a3406482838e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:36.186666 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127c6c7-a855-4c64-8e00-a3406482838e admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:36.186955 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127c6c7-a855-4c64-8e00-a3406482838e admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:36.187379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127c6c7-a855-4c64-8e00-a3406482838e admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:36.188085 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127c6c7-a855-4c64-8e00-a3406482838e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:36.190189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127c6c7-a855-4c64-8e00-a3406482838e admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:36.268628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1009/2011] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:36 2022] POST /identity/v3/users => generated 295 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:36.276930 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746d052-9115-4111-8252-206043936d36 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:36.290519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746d052-9115-4111-8252-206043936d36 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:36.296000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58ebf4-6181-4dcb-b604-89ffb7a6fac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:36.296851 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58ebf4-6181-4dcb-b604-89ffb7a6fac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:36.297209 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58ebf4-6181-4dcb-b604-89ffb7a6fac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:36.297560 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58ebf4-6181-4dcb-b604-89ffb7a6fac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:36.298385 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58ebf4-6181-4dcb-b604-89ffb7a6fac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:36.300093 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58ebf4-6181-4dcb-b604-89ffb7a6fac6 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:36.302709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:36.313329 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1003/2012] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:36.324474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127c6c7-a855-4c64-8e00-a3406482838e admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:36.338011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127c6c7-a855-4c64-8e00-a3406482838e admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:36.342966 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e28d9-5191-4e5c-908a-767a02f0e274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:36.343646 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e28d9-5191-4e5c-908a-767a02f0e274 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:36.344096 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e28d9-5191-4e5c-908a-767a02f0e274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:36.344373 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e28d9-5191-4e5c-908a-767a02f0e274 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:36.345115 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e28d9-5191-4e5c-908a-767a02f0e274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:36.346751 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e28d9-5191-4e5c-908a-767a02f0e274 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:36.358578 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1010/2013] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:36.361923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58ebf4-6181-4dcb-b604-89ffb7a6fac6 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:36.372716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58ebf4-6181-4dcb-b604-89ffb7a6fac6 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:36.377342 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c703f-e245-44fb-a1d0-18ad7b73b784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:36.378242 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c703f-e245-44fb-a1d0-18ad7b73b784 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:36.379449 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c703f-e245-44fb-a1d0-18ad7b73b784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:36.379995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c703f-e245-44fb-a1d0-18ad7b73b784 admin admin] PATH_INFO: `/v3/projects/6fe745378b0644509ed376022a1e6b59/users/49cb83a8cd3a4bf38f384da4ec323a56/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:36.396347 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c703f-e245-44fb-a1d0-18ad7b73b784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fe745378b0644509ed376022a1e6b59, user_id=49cb83a8cd3a4bf38f384da4ec323a56, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:36.399213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c703f-e245-44fb-a1d0-18ad7b73b784 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:36.500144 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1004/2014] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:36 2022] PUT /identity/v3/projects/6fe745378b0644509ed376022a1e6b59/users/49cb83a8cd3a4bf38f384da4ec323a56/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:36.512018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c449f6-1880-4700-85ed-5c3f3607100a None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:36.512310 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c449f6-1880-4700-85ed-5c3f3607100a None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:36.512588 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c449f6-1880-4700-85ed-5c3f3607100a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:36.538771 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-13c449f6-1880-4700-85ed-5c3f3607100a None None] MFA Rules not processed for user `49cb83a8cd3a4bf38f384da4ec323a56`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:36.539952 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-13c449f6-1880-4700-85ed-5c3f3607100a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:36.606641 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1011/2015] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:36.611239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c703f-e245-44fb-a1d0-18ad7b73b784 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:36.682350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c703f-e245-44fb-a1d0-18ad7b73b784 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:36.687607 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500019cf-daf8-4c65-bbf8-fccfc041de4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:36.688275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500019cf-daf8-4c65-bbf8-fccfc041de4f admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:36.688557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500019cf-daf8-4c65-bbf8-fccfc041de4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:36.688835 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500019cf-daf8-4c65-bbf8-fccfc041de4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:36.689536 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500019cf-daf8-4c65-bbf8-fccfc041de4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:36.692325 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500019cf-daf8-4c65-bbf8-fccfc041de4f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:36.781300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1005/2016] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:36 2022] POST /identity/v3/projects => generated 393 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:36.785538 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c449f6-1880-4700-85ed-5c3f3607100a None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:36.796194 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c449f6-1880-4700-85ed-5c3f3607100a None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:36.861856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992273d-9fc5-42f4-90d7-aa7fc4cef743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:36.871358 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992273d-9fc5-42f4-90d7-aa7fc4cef743 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:36.871741 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992273d-9fc5-42f4-90d7-aa7fc4cef743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:36.872095 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992273d-9fc5-42f4-90d7-aa7fc4cef743 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:36.873072 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992273d-9fc5-42f4-90d7-aa7fc4cef743 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:36.875066 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992273d-9fc5-42f4-90d7-aa7fc4cef743 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:36.940556 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1012/2017] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:36 2022] POST /identity/v3/users => generated 303 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:36.944914 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500019cf-daf8-4c65-bbf8-fccfc041de4f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:36.955289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500019cf-daf8-4c65-bbf8-fccfc041de4f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:36.960769 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f96f03-c9fe-4df5-95af-3aed935ac037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:36.961436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f96f03-c9fe-4df5-95af-3aed935ac037 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:36.961719 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f96f03-c9fe-4df5-95af-3aed935ac037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:36.962000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f96f03-c9fe-4df5-95af-3aed935ac037 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:36.962748 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f96f03-c9fe-4df5-95af-3aed935ac037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:36.964343 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f96f03-c9fe-4df5-95af-3aed935ac037 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:36.976572 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1006/2018] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:36.980691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992273d-9fc5-42f4-90d7-aa7fc4cef743 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:36.991686 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992273d-9fc5-42f4-90d7-aa7fc4cef743 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:36.996339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa234184-991e-46f4-8b15-94ba3e0167f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:36.997249 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa234184-991e-46f4-8b15-94ba3e0167f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:36.997536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa234184-991e-46f4-8b15-94ba3e0167f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:36.997811 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa234184-991e-46f4-8b15-94ba3e0167f4 admin admin] PATH_INFO: `/v3/projects/126d433f939d4bbd906494ab8dde9648/users/14e67fefe9844d64be9394ac18f65328/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:37.013535 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa234184-991e-46f4-8b15-94ba3e0167f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=126d433f939d4bbd906494ab8dde9648, user_id=14e67fefe9844d64be9394ac18f65328, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:37.016409 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa234184-991e-46f4-8b15-94ba3e0167f4 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:37.066800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:37.072391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1013/2019] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:36 2022] PUT /identity/v3/projects/126d433f939d4bbd906494ab8dde9648/users/14e67fefe9844d64be9394ac18f65328/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:37.076127 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f96f03-c9fe-4df5-95af-3aed935ac037 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:37.087299 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f96f03-c9fe-4df5-95af-3aed935ac037 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:37.146964 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40b95e-ca26-447a-ba0a-e63028673ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:37.147726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40b95e-ca26-447a-ba0a-e63028673ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:37.148007 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40b95e-ca26-447a-ba0a-e63028673ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:37.148282 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40b95e-ca26-447a-ba0a-e63028673ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:37.149005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40b95e-ca26-447a-ba0a-e63028673ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:37.150647 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40b95e-ca26-447a-ba0a-e63028673ae5 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:37.160636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1007/2020] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:37.164852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa234184-991e-46f4-8b15-94ba3e0167f4 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:37.175143 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa234184-991e-46f4-8b15-94ba3e0167f4 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:37.180726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2ae3f-a947-4632-9c2b-2ab516797f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:37.181598 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2ae3f-a947-4632-9c2b-2ab516797f81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:37.181881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2ae3f-a947-4632-9c2b-2ab516797f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:37.182155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2ae3f-a947-4632-9c2b-2ab516797f81 admin admin] PATH_INFO: `/v3/projects/126d433f939d4bbd906494ab8dde9648/users/14e67fefe9844d64be9394ac18f65328/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:37.184128 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2ae3f-a947-4632-9c2b-2ab516797f81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=126d433f939d4bbd906494ab8dde9648, user_id=14e67fefe9844d64be9394ac18f65328, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:37.186782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2ae3f-a947-4632-9c2b-2ab516797f81 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:37.198237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1014/2021] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:37 2022] PUT /identity/v3/projects/126d433f939d4bbd906494ab8dde9648/users/14e67fefe9844d64be9394ac18f65328/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:37.208687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644a8bd-1ef1-4061-ae1c-1c30ce059ac4 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:37.209228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644a8bd-1ef1-4061-ae1c-1c30ce059ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:37.209536 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644a8bd-1ef1-4061-ae1c-1c30ce059ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:37.235138 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-e644a8bd-1ef1-4061-ae1c-1c30ce059ac4 None None] MFA Rules not processed for user `14e67fefe9844d64be9394ac18f65328`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:37.236322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e644a8bd-1ef1-4061-ae1c-1c30ce059ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:37.270399 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:37.306894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1008/2022] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:37.318219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2ae3f-a947-4632-9c2b-2ab516797f81 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:37.319970 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-2ff2ae3f-a947-4632-9c2b-2ab516797f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:37.376924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2ae3f-a947-4632-9c2b-2ab516797f81 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:37.381192 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b859d88-e737-45c3-85f1-d7ac412af25f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:37.381836 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b859d88-e737-45c3-85f1-d7ac412af25f service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:37.382118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b859d88-e737-45c3-85f1-d7ac412af25f service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:37.382390 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b859d88-e737-45c3-85f1-d7ac412af25f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:37.391588 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b859d88-e737-45c3-85f1-d7ac412af25f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:37.393052 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b859d88-e737-45c3-85f1-d7ac412af25f service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:37.402651 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1015/2023] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:37 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:37.452878 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644a8bd-1ef1-4061-ae1c-1c30ce059ac4 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:37.454457 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e644a8bd-1ef1-4061-ae1c-1c30ce059ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:37.468102 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644a8bd-1ef1-4061-ae1c-1c30ce059ac4 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:37.471790 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1afd4c-f8ce-4289-ba22-160cba06c626 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:37.472958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1afd4c-f8ce-4289-ba22-160cba06c626 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:37.473603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1afd4c-f8ce-4289-ba22-160cba06c626 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:37.473603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1afd4c-f8ce-4289-ba22-160cba06c626 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:37.533207 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1afd4c-f8ce-4289-ba22-160cba06c626 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:37.534691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1afd4c-f8ce-4289-ba22-160cba06c626 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:37.544503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1009/2024] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:40.281868 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b859d88-e737-45c3-85f1-d7ac412af25f service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:40.281868 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-4b859d88-e737-45c3-85f1-d7ac412af25f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:40.361419 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b859d88-e737-45c3-85f1-d7ac412af25f service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:40.368708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca13212-a56b-4d56-acb7-9929ce3aa4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:40.369634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca13212-a56b-4d56-acb7-9929ce3aa4df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:40.369935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca13212-a56b-4d56-acb7-9929ce3aa4df admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:40.370198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca13212-a56b-4d56-acb7-9929ce3aa4df admin admin] PATH_INFO: `/v3/users/7b26b318fbef4edfb7e1caa5156e1bfc` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:40.371474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca13212-a56b-4d56-acb7-9929ce3aa4df admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b26b318fbef4edfb7e1caa5156e1bfc)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:40.373413 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca13212-a56b-4d56-acb7-9929ce3aa4df admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:40.559670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-eca13212-a56b-4d56-acb7-9929ce3aa4df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b26b318fbef4edfb7e1caa5156e1bfc'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:40.569243 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-eca13212-a56b-4d56-acb7-9929ce3aa4df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b26b318fbef4edfb7e1caa5156e1bfc'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:40.570083 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-eca13212-a56b-4d56-acb7-9929ce3aa4df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b26b318fbef4edfb7e1caa5156e1bfc'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:40.571126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-eca13212-a56b-4d56-acb7-9929ce3aa4df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b26b318fbef4edfb7e1caa5156e1bfc'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:40.596134 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-eca13212-a56b-4d56-acb7-9929ce3aa4df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b26b318fbef4edfb7e1caa5156e1bfc'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:40.666752 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1016/2025] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:40 2022] DELETE /identity/v3/users/7b26b318fbef4edfb7e1caa5156e1bfc => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:40.673103 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8163c75-8048-417e-b6c4-f09a14338b9c None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:40.673579 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8163c75-8048-417e-b6c4-f09a14338b9c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:40.673671 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8163c75-8048-417e-b6c4-f09a14338b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:40.693333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f8163c75-8048-417e-b6c4-f09a14338b9c None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:40.694751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f8163c75-8048-417e-b6c4-f09a14338b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:40.763299 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1010/2026] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:40.774636 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca13212-a56b-4d56-acb7-9929ce3aa4df admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:40.775970 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-eca13212-a56b-4d56-acb7-9929ce3aa4df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:40.785728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:40.833854 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca13212-a56b-4d56-acb7-9929ce3aa4df admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:40.837987 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40708c06-78ef-4990-92ad-c7e88b949eae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:40.838697 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40708c06-78ef-4990-92ad-c7e88b949eae service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:40.838979 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40708c06-78ef-4990-92ad-c7e88b949eae service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:40.839257 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40708c06-78ef-4990-92ad-c7e88b949eae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:40.848921 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40708c06-78ef-4990-92ad-c7e88b949eae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:40.850425 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40708c06-78ef-4990-92ad-c7e88b949eae service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:40.861528 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1017/2027] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:40.907318 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8163c75-8048-417e-b6c4-f09a14338b9c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:40.909493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f8163c75-8048-417e-b6c4-f09a14338b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:40.928745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8163c75-8048-417e-b6c4-f09a14338b9c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:40.937324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea6944-e8ad-452e-b85c-52eb26eb3296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:40.938895 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea6944-e8ad-452e-b85c-52eb26eb3296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:40.939372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea6944-e8ad-452e-b85c-52eb26eb3296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:40.940074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea6944-e8ad-452e-b85c-52eb26eb3296 admin admin] PATH_INFO: `/v3/users/bea62fd563dc424cb9beee5a5b1acb52` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:40.942269 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea6944-e8ad-452e-b85c-52eb26eb3296 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bea62fd563dc424cb9beee5a5b1acb52)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:40.946511 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea6944-e8ad-452e-b85c-52eb26eb3296 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:41.212723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-95ea6944-e8ad-452e-b85c-52eb26eb3296 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bea62fd563dc424cb9beee5a5b1acb52'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:41.228099 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-95ea6944-e8ad-452e-b85c-52eb26eb3296 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bea62fd563dc424cb9beee5a5b1acb52'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:41.228933 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-95ea6944-e8ad-452e-b85c-52eb26eb3296 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bea62fd563dc424cb9beee5a5b1acb52'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:41.229819 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-95ea6944-e8ad-452e-b85c-52eb26eb3296 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bea62fd563dc424cb9beee5a5b1acb52'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:41.237132 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:41.246970 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-95ea6944-e8ad-452e-b85c-52eb26eb3296 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bea62fd563dc424cb9beee5a5b1acb52'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:41.346144 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1011/2028] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:40 2022] DELETE /identity/v3/users/bea62fd563dc424cb9beee5a5b1acb52 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:41.352051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb3c9c-078f-4328-a8be-11a0caf6a476 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:41.352521 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb3c9c-078f-4328-a8be-11a0caf6a476 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:41.352622 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb3c9c-078f-4328-a8be-11a0caf6a476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:41.371976 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c3fb3c9c-078f-4328-a8be-11a0caf6a476 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:41.373192 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c3fb3c9c-078f-4328-a8be-11a0caf6a476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:41.398364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea6944-e8ad-452e-b85c-52eb26eb3296 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:41.399877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-95ea6944-e8ad-452e-b85c-52eb26eb3296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:41.504428 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1018/2029] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:41.511028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb3c9c-078f-4328-a8be-11a0caf6a476 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:41.512948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c3fb3c9c-078f-4328-a8be-11a0caf6a476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:41.529024 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea6944-e8ad-452e-b85c-52eb26eb3296 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:41.534021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a38ea4-df54-4152-91d6-dabd3cd8aab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:41.535119 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a38ea4-df54-4152-91d6-dabd3cd8aab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:41.535415 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a38ea4-df54-4152-91d6-dabd3cd8aab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:41.538180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a38ea4-df54-4152-91d6-dabd3cd8aab4 admin admin] PATH_INFO: `/v3/projects/8333f0921023477fa9470a40e3cca03c` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:41.538180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a38ea4-df54-4152-91d6-dabd3cd8aab4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8333f0921023477fa9470a40e3cca03c)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:41.539881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a38ea4-df54-4152-91d6-dabd3cd8aab4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:41.597974 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb3c9c-078f-4328-a8be-11a0caf6a476 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:41.604254 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc409a-4221-41bb-a287-66d6938c2069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:41.604953 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc409a-4221-41bb-a287-66d6938c2069 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:41.605192 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc409a-4221-41bb-a287-66d6938c2069 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:41.605904 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc409a-4221-41bb-a287-66d6938c2069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:41.616796 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc409a-4221-41bb-a287-66d6938c2069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:41.618582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc409a-4221-41bb-a287-66d6938c2069 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:41.631750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1019/2030] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:45:41.670869 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b8a38ea4-df54-4152-91d6-dabd3cd8aab4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8333f0921023477fa9470a40e3cca03c', 'request_id': 'req-b8a38ea4-df54-4152-91d6-dabd3cd8aab4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:41.679186 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b8a38ea4-df54-4152-91d6-dabd3cd8aab4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8333f0921023477fa9470a40e3cca03c', 'request_id': 'req-b8a38ea4-df54-4152-91d6-dabd3cd8aab4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:41.800979 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b8a38ea4-df54-4152-91d6-dabd3cd8aab4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8333f0921023477fa9470a40e3cca03c', 'request_id': 'req-b8a38ea4-df54-4152-91d6-dabd3cd8aab4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:41.812832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b8a38ea4-df54-4152-91d6-dabd3cd8aab4 admin admin] The token cache is being invalidate because project 8333f0921023477fa9470a40e3cca03c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:41.813119 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b8a38ea4-df54-4152-91d6-dabd3cd8aab4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:41.813782 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b8a38ea4-df54-4152-91d6-dabd3cd8aab4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:41.815386 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1012/2031] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:41 2022] DELETE /identity/v3/projects/8333f0921023477fa9470a40e3cca03c => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:41.819848 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc409a-4221-41bb-a287-66d6938c2069 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:41.829896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc409a-4221-41bb-a287-66d6938c2069 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:41.886485 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:41.915584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d9758-f60f-4b39-8d55-ccbaf1c7e75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:41.917538 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d9758-f60f-4b39-8d55-ccbaf1c7e75e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:41.918373 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d9758-f60f-4b39-8d55-ccbaf1c7e75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:41.919127 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d9758-f60f-4b39-8d55-ccbaf1c7e75e admin admin] PATH_INFO: `/v3/users/68c62a4b4a9845d9bdc10f9f8afc7a13` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:41.921369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d9758-f60f-4b39-8d55-ccbaf1c7e75e admin admin] RBAC: Authorizing `identity:delete_user(user_id=68c62a4b4a9845d9bdc10f9f8afc7a13)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:41.923872 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d9758-f60f-4b39-8d55-ccbaf1c7e75e admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:42.148882 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-682d9758-f60f-4b39-8d55-ccbaf1c7e75e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68c62a4b4a9845d9bdc10f9f8afc7a13'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:42.157242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-682d9758-f60f-4b39-8d55-ccbaf1c7e75e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68c62a4b4a9845d9bdc10f9f8afc7a13'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:42.157969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-682d9758-f60f-4b39-8d55-ccbaf1c7e75e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68c62a4b4a9845d9bdc10f9f8afc7a13'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:42.158635 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-682d9758-f60f-4b39-8d55-ccbaf1c7e75e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68c62a4b4a9845d9bdc10f9f8afc7a13'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:42.173574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a38ea4-df54-4152-91d6-dabd3cd8aab4 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:42.177110 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b8a38ea4-df54-4152-91d6-dabd3cd8aab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:42.198351 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-682d9758-f60f-4b39-8d55-ccbaf1c7e75e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68c62a4b4a9845d9bdc10f9f8afc7a13'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:42.227825 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a38ea4-df54-4152-91d6-dabd3cd8aab4 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:42.240239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80863830-7da1-489d-a2be-6f25eb840e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:42.241477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80863830-7da1-489d-a2be-6f25eb840e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:42.241773 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80863830-7da1-489d-a2be-6f25eb840e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:42.242059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80863830-7da1-489d-a2be-6f25eb840e71 admin admin] PATH_INFO: `/v3/projects/abb8c04ae0ec4a948826540a2d5e947d` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:42.245528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80863830-7da1-489d-a2be-6f25eb840e71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abb8c04ae0ec4a948826540a2d5e947d)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:42.248217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80863830-7da1-489d-a2be-6f25eb840e71 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:42.300007 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1020/2032] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:41 2022] DELETE /identity/v3/users/68c62a4b4a9845d9bdc10f9f8afc7a13 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:42.393935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-80863830-7da1-489d-a2be-6f25eb840e71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abb8c04ae0ec4a948826540a2d5e947d', 'request_id': 'req-80863830-7da1-489d-a2be-6f25eb840e71'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:42.400357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-80863830-7da1-489d-a2be-6f25eb840e71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abb8c04ae0ec4a948826540a2d5e947d', 'request_id': 'req-80863830-7da1-489d-a2be-6f25eb840e71'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:42.516029 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-80863830-7da1-489d-a2be-6f25eb840e71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abb8c04ae0ec4a948826540a2d5e947d', 'request_id': 'req-80863830-7da1-489d-a2be-6f25eb840e71'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:42.521436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:42.530856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-80863830-7da1-489d-a2be-6f25eb840e71 admin admin] The token cache is being invalidate because project abb8c04ae0ec4a948826540a2d5e947d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:42.531392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-80863830-7da1-489d-a2be-6f25eb840e71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:42.532810 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-80863830-7da1-489d-a2be-6f25eb840e71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:42.535029 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1013/2033] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:42 2022] DELETE /identity/v3/projects/abb8c04ae0ec4a948826540a2d5e947d => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:42.539982 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c8c46-38e3-457e-b498-9d15a956f092 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:42.540472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c8c46-38e3-457e-b498-9d15a956f092 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:42.540547 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c8c46-38e3-457e-b498-9d15a956f092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:42.572807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-7a3c8c46-38e3-457e-b498-9d15a956f092 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:42.574709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-7a3c8c46-38e3-457e-b498-9d15a956f092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:42.655964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1021/2034] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:42.657808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a74679-cf6a-48c2-aa8a-52ec6b591691 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:42.658742 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a74679-cf6a-48c2-aa8a-52ec6b591691 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:42.659042 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a74679-cf6a-48c2-aa8a-52ec6b591691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:42.680993 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-83a74679-cf6a-48c2-aa8a-52ec6b591691 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:42.682525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-83a74679-cf6a-48c2-aa8a-52ec6b591691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:42.694413 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1014/2035] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:42.698903 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c8c46-38e3-457e-b498-9d15a956f092 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:42.709873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c8c46-38e3-457e-b498-9d15a956f092 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:42.714791 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b1c4c-2cd6-4750-b4df-d146ebd892a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:42.715462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b1c4c-2cd6-4750-b4df-d146ebd892a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:42.715746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b1c4c-2cd6-4750-b4df-d146ebd892a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:42.716022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b1c4c-2cd6-4750-b4df-d146ebd892a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:42.716765 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b1c4c-2cd6-4750-b4df-d146ebd892a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:42.717063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b1c4c-2cd6-4750-b4df-d146ebd892a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:42.718691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b1c4c-2cd6-4750-b4df-d146ebd892a6 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:42.734582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1022/2036] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:42.736287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a74679-cf6a-48c2-aa8a-52ec6b591691 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:42.746338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a74679-cf6a-48c2-aa8a-52ec6b591691 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:42.751475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0967b-a324-4def-ac8d-8ce131f78b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:42.752141 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0967b-a324-4def-ac8d-8ce131f78b49 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:42.752428 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0967b-a324-4def-ac8d-8ce131f78b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:42.752705 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0967b-a324-4def-ac8d-8ce131f78b49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:42.753409 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0967b-a324-4def-ac8d-8ce131f78b49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:42.755491 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0967b-a324-4def-ac8d-8ce131f78b49 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:42.810735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b1c4c-2cd6-4750-b4df-d146ebd892a6 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:42.813495 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-1e1b1c4c-2cd6-4750-b4df-d146ebd892a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:42.835423 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b1c4c-2cd6-4750-b4df-d146ebd892a6 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:42.844068 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0ec4a-4b4b-4530-a0b2-b72af839148c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:42.845646 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0ec4a-4b4b-4530-a0b2-b72af839148c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:42.849182 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0ec4a-4b4b-4530-a0b2-b72af839148c admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:42.849182 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0ec4a-4b4b-4530-a0b2-b72af839148c admin admin] PATH_INFO: `/v3/projects/823eb0bf1d4e4afc876da6311f40ca55` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:42.849377 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0ec4a-4b4b-4530-a0b2-b72af839148c admin admin] RBAC: Authorizing `identity:delete_project(project_id=823eb0bf1d4e4afc876da6311f40ca55)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:42.853347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0ec4a-4b4b-4530-a0b2-b72af839148c admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:42.880065 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1015/2037] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:42 2022] POST /identity/v3/projects => generated 375 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:42.886863 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0967b-a324-4def-ac8d-8ce131f78b49 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:42.970363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0967b-a324-4def-ac8d-8ce131f78b49 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:42.978096 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a5efa-e0f1-4107-bcdd-21271ec97a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:42.979002 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a5efa-e0f1-4107-bcdd-21271ec97a04 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:42.979381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a5efa-e0f1-4107-bcdd-21271ec97a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:42.979750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a5efa-e0f1-4107-bcdd-21271ec97a04 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:42.980601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a5efa-e0f1-4107-bcdd-21271ec97a04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:42.983373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a5efa-e0f1-4107-bcdd-21271ec97a04 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:43.063276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-b5c0ec4a-4b4b-4530-a0b2-b72af839148c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '823eb0bf1d4e4afc876da6311f40ca55', 'request_id': 'req-b5c0ec4a-4b4b-4530-a0b2-b72af839148c'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:43.072239 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-b5c0ec4a-4b4b-4530-a0b2-b72af839148c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '823eb0bf1d4e4afc876da6311f40ca55', 'request_id': 'req-b5c0ec4a-4b4b-4530-a0b2-b72af839148c'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:43.074146 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:43.088794 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1016/2038] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:42 2022] POST /identity/v3/users => generated 288 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:43.092781 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a5efa-e0f1-4107-bcdd-21271ec97a04 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:43.106882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a5efa-e0f1-4107-bcdd-21271ec97a04 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:43.156720 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-b5c0ec4a-4b4b-4530-a0b2-b72af839148c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '823eb0bf1d4e4afc876da6311f40ca55', 'request_id': 'req-b5c0ec4a-4b4b-4530-a0b2-b72af839148c'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:43.173189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-b5c0ec4a-4b4b-4530-a0b2-b72af839148c admin admin] The token cache is being invalidate because project 823eb0bf1d4e4afc876da6311f40ca55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:43.173378 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-b5c0ec4a-4b4b-4530-a0b2-b72af839148c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:43.174167 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-b5c0ec4a-4b4b-4530-a0b2-b72af839148c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:43.177709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1023/2039] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:42 2022] DELETE /identity/v3/projects/823eb0bf1d4e4afc876da6311f40ca55 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:45:43.184520 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e61b4-87a1-48c0-9a3f-c811fbe365b0 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:43.185044 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e61b4-87a1-48c0-9a3f-c811fbe365b0 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:43.186253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e61b4-87a1-48c0-9a3f-c811fbe365b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:43.189360 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576dcdf-5259-4ba4-8048-9338adb935d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:43.190041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576dcdf-5259-4ba4-8048-9338adb935d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:43.190326 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576dcdf-5259-4ba4-8048-9338adb935d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:43.190633 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576dcdf-5259-4ba4-8048-9338adb935d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:43.191426 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576dcdf-5259-4ba4-8048-9338adb935d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:43.193834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576dcdf-5259-4ba4-8048-9338adb935d0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:43.207121 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1017/2040] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:43.211307 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576dcdf-5259-4ba4-8048-9338adb935d0 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:43.212283 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c08e61b4-87a1-48c0-9a3f-c811fbe365b0 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:43.213161 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1576dcdf-5259-4ba4-8048-9338adb935d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:43.214511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c08e61b4-87a1-48c0-9a3f-c811fbe365b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:43.225420 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1024/2041] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:45:43.237225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e8561-63a6-4739-85f3-5e7091729f6c None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:43.237625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e8561-63a6-4739-85f3-5e7091729f6c None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:43.238094 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e8561-63a6-4739-85f3-5e7091729f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:43.243741 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576dcdf-5259-4ba4-8048-9338adb935d0 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:43.256263 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea2ffd-7caf-4b32-b2d7-591489b05595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:43.260503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea2ffd-7caf-4b32-b2d7-591489b05595 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:43.262007 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea2ffd-7caf-4b32-b2d7-591489b05595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:43.262439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea2ffd-7caf-4b32-b2d7-591489b05595 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:43.263376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dea2ffd-7caf-4b32-b2d7-591489b05595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:43.275768 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dea2ffd-7caf-4b32-b2d7-591489b05595 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:43.283916 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-8e1e8561-63a6-4739-85f3-5e7091729f6c None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:43.289102 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-8e1e8561-63a6-4739-85f3-5e7091729f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:43.294002 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1018/2042] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:43.296445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea2ffd-7caf-4b32-b2d7-591489b05595 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:43.311561 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1025/2043] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:43.320020 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea2ffd-7caf-4b32-b2d7-591489b05595 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:43.321062 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e8561-63a6-4739-85f3-5e7091729f6c None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:43.328038 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82694e44-e216-4641-8f12-84af96f52840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:43.329006 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82694e44-e216-4641-8f12-84af96f52840 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:43.329285 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82694e44-e216-4641-8f12-84af96f52840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:43.329605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82694e44-e216-4641-8f12-84af96f52840 admin admin] PATH_INFO: `/v3/projects/ae538de4a6f4486d84e8595d564872f1/users/568a2758c088444ba34f1dcba8a981cb/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:43.333479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e8561-63a6-4739-85f3-5e7091729f6c None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:43.347314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f728796-392c-46e7-b15e-d3f3aa1172b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:43.347703 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82694e44-e216-4641-8f12-84af96f52840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae538de4a6f4486d84e8595d564872f1, user_id=568a2758c088444ba34f1dcba8a981cb, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:43.351727 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f728796-392c-46e7-b15e-d3f3aa1172b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:43.351727 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f728796-392c-46e7-b15e-d3f3aa1172b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:43.351727 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f728796-392c-46e7-b15e-d3f3aa1172b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:43.351968 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82694e44-e216-4641-8f12-84af96f52840 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:43.352042 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f728796-392c-46e7-b15e-d3f3aa1172b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:43.352042 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f728796-392c-46e7-b15e-d3f3aa1172b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:43.368270 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f728796-392c-46e7-b15e-d3f3aa1172b8 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:43.382727 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1026/2044] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:45:43.387023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f728796-392c-46e7-b15e-d3f3aa1172b8 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:43.398617 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f728796-392c-46e7-b15e-d3f3aa1172b8 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:43.404434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99b4b0-4b22-439a-9b38-5979f10a6169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:43.405923 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99b4b0-4b22-439a-9b38-5979f10a6169 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:43.406333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99b4b0-4b22-439a-9b38-5979f10a6169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:43.406720 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99b4b0-4b22-439a-9b38-5979f10a6169 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:43.408141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99b4b0-4b22-439a-9b38-5979f10a6169 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:43.410295 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99b4b0-4b22-439a-9b38-5979f10a6169 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:43.444331 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1019/2045] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:43 2022] PUT /identity/v3/projects/ae538de4a6f4486d84e8595d564872f1/users/568a2758c088444ba34f1dcba8a981cb/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:45:43.458607 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5318a50-d32b-4633-904e-3a1615587099 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:43.459461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5318a50-d32b-4633-904e-3a1615587099 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:43.459917 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5318a50-d32b-4633-904e-3a1615587099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:43.494246 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d5318a50-d32b-4633-904e-3a1615587099 None None] MFA Rules not processed for user `568a2758c088444ba34f1dcba8a981cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:43.495717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d5318a50-d32b-4633-904e-3a1615587099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:43.502856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:43.541091 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1027/2046] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:43 2022] POST /identity/v3/projects => generated 407 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:45:43.542320 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99b4b0-4b22-439a-9b38-5979f10a6169 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:43.635533 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1020/2047] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:45:43.651862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99b4b0-4b22-439a-9b38-5979f10a6169 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:43.657327 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7422df4-df9c-4dde-9f08-e6697e358f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:43.658182 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7422df4-df9c-4dde-9f08-e6697e358f77 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:43.658307 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7422df4-df9c-4dde-9f08-e6697e358f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:43.658556 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7422df4-df9c-4dde-9f08-e6697e358f77 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:43.659268 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7422df4-df9c-4dde-9f08-e6697e358f77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:43.661744 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7422df4-df9c-4dde-9f08-e6697e358f77 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:43.740675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5318a50-d32b-4633-904e-3a1615587099 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:43.742159 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d5318a50-d32b-4633-904e-3a1615587099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:43.772716 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1028/2048] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:43 2022] POST /identity/v3/users => generated 304 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:45:43.772716 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7422df4-df9c-4dde-9f08-e6697e358f77 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:43.774852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a7422df4-df9c-4dde-9f08-e6697e358f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:43.837393 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5318a50-d32b-4633-904e-3a1615587099 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:43.842874 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3ae97-e1d7-4121-b574-b006fdcf4cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:43.843777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3ae97-e1d7-4121-b574-b006fdcf4cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:43.844222 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3ae97-e1d7-4121-b574-b006fdcf4cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:43.844584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3ae97-e1d7-4121-b574-b006fdcf4cd2 admin admin] PATH_INFO: `/v3/users/49cb83a8cd3a4bf38f384da4ec323a56` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:43.846073 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3ae97-e1d7-4121-b574-b006fdcf4cd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49cb83a8cd3a4bf38f384da4ec323a56)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:43.853542 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3ae97-e1d7-4121-b574-b006fdcf4cd2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:43.885115 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7422df4-df9c-4dde-9f08-e6697e358f77 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:43.889062 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56dba0b-3592-40e3-b9fd-4de0e40ba435 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:43.889712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dba0b-3592-40e3-b9fd-4de0e40ba435 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:43.890016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dba0b-3592-40e3-b9fd-4de0e40ba435 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:43.890294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dba0b-3592-40e3-b9fd-4de0e40ba435 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:43.962030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56dba0b-3592-40e3-b9fd-4de0e40ba435 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:43.963538 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56dba0b-3592-40e3-b9fd-4de0e40ba435 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:43.975774 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1029/2049] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:43.976974 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56dba0b-3592-40e3-b9fd-4de0e40ba435 service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:43.987839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:43.994505 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56dba0b-3592-40e3-b9fd-4de0e40ba435 service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:44.000009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5035366-0069-4497-ba84-152921e27e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:44.000904 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5035366-0069-4497-ba84-152921e27e32 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:44.001193 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5035366-0069-4497-ba84-152921e27e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:44.001471 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5035366-0069-4497-ba84-152921e27e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:44.002209 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5035366-0069-4497-ba84-152921e27e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:44.004055 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5035366-0069-4497-ba84-152921e27e32 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:44.022035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1030/2050] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:44.023484 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5035366-0069-4497-ba84-152921e27e32 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:44.041592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5035366-0069-4497-ba84-152921e27e32 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:44.047647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafee87e-dc8f-43a5-9d56-a03cc3740247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:44.048448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafee87e-dc8f-43a5-9d56-a03cc3740247 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:44.048795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafee87e-dc8f-43a5-9d56-a03cc3740247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:44.050909 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafee87e-dc8f-43a5-9d56-a03cc3740247 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:44.050909 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafee87e-dc8f-43a5-9d56-a03cc3740247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:44.052391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafee87e-dc8f-43a5-9d56-a03cc3740247 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:44.063151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1031/2051] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:44.067023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafee87e-dc8f-43a5-9d56-a03cc3740247 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:44.077937 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafee87e-dc8f-43a5-9d56-a03cc3740247 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:44.084644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42254280-9871-417f-b9b5-aa0ba389dafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:44.085541 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42254280-9871-417f-b9b5-aa0ba389dafb admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:44.085825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42254280-9871-417f-b9b5-aa0ba389dafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:44.086099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42254280-9871-417f-b9b5-aa0ba389dafb admin admin] PATH_INFO: `/v3/projects/44105066239048dcb849ae61eb73c150/users/f0cd4c9d85854f7f83aa56cbaaabd798/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:44.096531 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7cd3ae97-e1d7-4121-b574-b006fdcf4cd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49cb83a8cd3a4bf38f384da4ec323a56'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:44.104244 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42254280-9871-417f-b9b5-aa0ba389dafb admin admin] RBAC: Authorizing `identity:create_grant(project_id=44105066239048dcb849ae61eb73c150, user_id=f0cd4c9d85854f7f83aa56cbaaabd798, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:44.104805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7cd3ae97-e1d7-4121-b574-b006fdcf4cd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49cb83a8cd3a4bf38f384da4ec323a56'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:44.105714 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7cd3ae97-e1d7-4121-b574-b006fdcf4cd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49cb83a8cd3a4bf38f384da4ec323a56'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:44.106616 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7cd3ae97-e1d7-4121-b574-b006fdcf4cd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49cb83a8cd3a4bf38f384da4ec323a56'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:44.106961 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42254280-9871-417f-b9b5-aa0ba389dafb admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:44.121145 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7cd3ae97-e1d7-4121-b574-b006fdcf4cd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49cb83a8cd3a4bf38f384da4ec323a56'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:44.185620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1032/2052] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:44 2022] PUT /identity/v3/projects/44105066239048dcb849ae61eb73c150/users/f0cd4c9d85854f7f83aa56cbaaabd798/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:45:44.197221 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b8a62-d9ff-4ccc-a8e6-f10867f337fb None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:44.197512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b8a62-d9ff-4ccc-a8e6-f10867f337fb None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:44.197794 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b8a62-d9ff-4ccc-a8e6-f10867f337fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:44.210785 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1021/2053] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:43 2022] DELETE /identity/v3/users/49cb83a8cd3a4bf38f384da4ec323a56 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:44.215723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b560b8f-bf8d-4792-92ac-86b76b97f3ef None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:44.216012 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b560b8f-bf8d-4792-92ac-86b76b97f3ef None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:44.216290 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b560b8f-bf8d-4792-92ac-86b76b97f3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:44.231667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-a86b8a62-d9ff-4ccc-a8e6-f10867f337fb None None] MFA Rules not processed for user `f0cd4c9d85854f7f83aa56cbaaabd798`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:44.232889 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a86b8a62-d9ff-4ccc-a8e6-f10867f337fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:44.252093 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-9b560b8f-bf8d-4792-92ac-86b76b97f3ef None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:44.253744 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9b560b8f-bf8d-4792-92ac-86b76b97f3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:44.315182 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1033/2054] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:44 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:45:44.320455 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b8a62-d9ff-4ccc-a8e6-f10867f337fb None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:44.321880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a86b8a62-d9ff-4ccc-a8e6-f10867f337fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:44.326467 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1022/2055] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:44.336581 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b8a62-d9ff-4ccc-a8e6-f10867f337fb None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:44.337953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b560b8f-bf8d-4792-92ac-86b76b97f3ef None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:44.340502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9b560b8f-bf8d-4792-92ac-86b76b97f3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:44.342067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4d489-173d-4e38-96ea-b7c322ceb2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:44.342844 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4d489-173d-4e38-96ea-b7c322ceb2e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:44.343199 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4d489-173d-4e38-96ea-b7c322ceb2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:44.343576 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4d489-173d-4e38-96ea-b7c322ceb2e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:44.344180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4d489-173d-4e38-96ea-b7c322ceb2e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:44.346113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4d489-173d-4e38-96ea-b7c322ceb2e7 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:44.401439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b560b8f-bf8d-4792-92ac-86b76b97f3ef None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:44.405223 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f68d7-2e54-4d2d-bf05-add751e75fd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:44.405908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f68d7-2e54-4d2d-bf05-add751e75fd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:44.406191 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f68d7-2e54-4d2d-bf05-add751e75fd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:44.406626 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f68d7-2e54-4d2d-bf05-add751e75fd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:44.416755 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f68d7-2e54-4d2d-bf05-add751e75fd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:44.418225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f68d7-2e54-4d2d-bf05-add751e75fd5 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:44.428212 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1023/2056] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:45:44.485981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1034/2057] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:44 2022] POST /identity/v3/projects => generated 407 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:44.492234 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f68d7-2e54-4d2d-bf05-add751e75fd5 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:44.507101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:44.512790 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f68d7-2e54-4d2d-bf05-add751e75fd5 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:44.578609 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059f977-c959-4825-8f30-09bc3c1bb214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:44.580278 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059f977-c959-4825-8f30-09bc3c1bb214 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:44.580278 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059f977-c959-4825-8f30-09bc3c1bb214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:44.580278 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059f977-c959-4825-8f30-09bc3c1bb214 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:44.581048 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059f977-c959-4825-8f30-09bc3c1bb214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:44.583608 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059f977-c959-4825-8f30-09bc3c1bb214 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:44.659355 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1024/2058] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:44 2022] POST /identity/v3/users => generated 310 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:44.664364 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4d489-173d-4e38-96ea-b7c322ceb2e7 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:44.674767 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4d489-173d-4e38-96ea-b7c322ceb2e7 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:44.681568 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97256ce5-7089-48f7-91cf-caf6e6cb1103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:44.682246 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97256ce5-7089-48f7-91cf-caf6e6cb1103 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:44.682549 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97256ce5-7089-48f7-91cf-caf6e6cb1103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:44.683208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97256ce5-7089-48f7-91cf-caf6e6cb1103 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:44.683954 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97256ce5-7089-48f7-91cf-caf6e6cb1103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:44.685568 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97256ce5-7089-48f7-91cf-caf6e6cb1103 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:44.696782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1035/2059] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:44.701262 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059f977-c959-4825-8f30-09bc3c1bb214 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:44.712224 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059f977-c959-4825-8f30-09bc3c1bb214 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:44.717088 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff2663-f470-4619-a912-6cfa04e6704c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:44.718144 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff2663-f470-4619-a912-6cfa04e6704c admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:44.718428 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff2663-f470-4619-a912-6cfa04e6704c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:44.718750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff2663-f470-4619-a912-6cfa04e6704c admin admin] PATH_INFO: `/v3/projects/c04f546dbed548ad9c7b90782808c06b/users/e249d8a4289c4e3cb3cf9899169f8a48/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:44.740604 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff2663-f470-4619-a912-6cfa04e6704c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c04f546dbed548ad9c7b90782808c06b, user_id=e249d8a4289c4e3cb3cf9899169f8a48, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:44.743438 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff2663-f470-4619-a912-6cfa04e6704c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:44.838130 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1025/2060] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:44 2022] PUT /identity/v3/projects/c04f546dbed548ad9c7b90782808c06b/users/e249d8a4289c4e3cb3cf9899169f8a48/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:44.842511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97256ce5-7089-48f7-91cf-caf6e6cb1103 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:44.853153 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97256ce5-7089-48f7-91cf-caf6e6cb1103 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:44.898511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:44.910971 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa888e5-cf8a-450a-952a-6d708d8306d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:44.912350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa888e5-cf8a-450a-952a-6d708d8306d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:44.912633 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa888e5-cf8a-450a-952a-6d708d8306d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:44.912903 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa888e5-cf8a-450a-952a-6d708d8306d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:44.913628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa888e5-cf8a-450a-952a-6d708d8306d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:44.915317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa888e5-cf8a-450a-952a-6d708d8306d3 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:44.924836 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1036/2061] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:45:44.929295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff2663-f470-4619-a912-6cfa04e6704c admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:44.941188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff2663-f470-4619-a912-6cfa04e6704c admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:44.947010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe82633-ee91-4b47-8f33-19443ee7e398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:44.948056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe82633-ee91-4b47-8f33-19443ee7e398 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:44.948438 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe82633-ee91-4b47-8f33-19443ee7e398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:44.948811 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe82633-ee91-4b47-8f33-19443ee7e398 admin admin] PATH_INFO: `/v3/projects/c04f546dbed548ad9c7b90782808c06b/users/e249d8a4289c4e3cb3cf9899169f8a48/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:44.951517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe82633-ee91-4b47-8f33-19443ee7e398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c04f546dbed548ad9c7b90782808c06b, user_id=e249d8a4289c4e3cb3cf9899169f8a48, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:44.954383 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe82633-ee91-4b47-8f33-19443ee7e398 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:44.972718 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa888e5-cf8a-450a-952a-6d708d8306d3 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:44.975480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-1aa888e5-cf8a-450a-952a-6d708d8306d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:45.109971 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa888e5-cf8a-450a-952a-6d708d8306d3 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:45.111047 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1026/2062] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:44 2022] PUT /identity/v3/projects/c04f546dbed548ad9c7b90782808c06b/users/e249d8a4289c4e3cb3cf9899169f8a48/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:45.111047 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49703cf9-2983-4b27-b0a9-2be81c43bfe8 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:45.111047 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49703cf9-2983-4b27-b0a9-2be81c43bfe8 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:45.111047 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49703cf9-2983-4b27-b0a9-2be81c43bfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:45.111047 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-49703cf9-2983-4b27-b0a9-2be81c43bfe8 None None] MFA Rules not processed for user `e249d8a4289c4e3cb3cf9899169f8a48`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:45.111947 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-49703cf9-2983-4b27-b0a9-2be81c43bfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:45.112338 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6a3be-a352-49b9-af76-6a1021a5c361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:45.112338 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6a3be-a352-49b9-af76-6a1021a5c361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:45.112338 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6a3be-a352-49b9-af76-6a1021a5c361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:45.112338 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6a3be-a352-49b9-af76-6a1021a5c361 admin admin] PATH_INFO: `/v3/projects/6fe745378b0644509ed376022a1e6b59` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:45.112849 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6a3be-a352-49b9-af76-6a1021a5c361 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fe745378b0644509ed376022a1e6b59)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:45.112849 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6a3be-a352-49b9-af76-6a1021a5c361 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:45.141570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1027/2063] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:45 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:45:45.152055 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49703cf9-2983-4b27-b0a9-2be81c43bfe8 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:45.158472 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-49703cf9-2983-4b27-b0a9-2be81c43bfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:45.237030 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49703cf9-2983-4b27-b0a9-2be81c43bfe8 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:45.243117 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a93a5-ca23-4caf-8774-7ad5b7228f0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:45.243795 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a93a5-ca23-4caf-8774-7ad5b7228f0e service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:45.244112 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a93a5-ca23-4caf-8774-7ad5b7228f0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:45.244356 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a93a5-ca23-4caf-8774-7ad5b7228f0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:45.255412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a93a5-ca23-4caf-8774-7ad5b7228f0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:45.256856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a93a5-ca23-4caf-8774-7ad5b7228f0e service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:45.267098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1028/2064] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:45 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:45:45.346583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-47c6a3be-a352-49b9-af76-6a1021a5c361 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fe745378b0644509ed376022a1e6b59', 'request_id': 'req-47c6a3be-a352-49b9-af76-6a1021a5c361'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:45.353302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-47c6a3be-a352-49b9-af76-6a1021a5c361 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fe745378b0644509ed376022a1e6b59', 'request_id': 'req-47c6a3be-a352-49b9-af76-6a1021a5c361'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:45.430395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-47c6a3be-a352-49b9-af76-6a1021a5c361 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fe745378b0644509ed376022a1e6b59', 'request_id': 'req-47c6a3be-a352-49b9-af76-6a1021a5c361'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:45.434162 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:45.442050 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-47c6a3be-a352-49b9-af76-6a1021a5c361 admin admin] The token cache is being invalidate because project 6fe745378b0644509ed376022a1e6b59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:45.442403 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-47c6a3be-a352-49b9-af76-6a1021a5c361 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:45.443157 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-47c6a3be-a352-49b9-af76-6a1021a5c361 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:45.444880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1037/2065] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:44 2022] DELETE /identity/v3/projects/6fe745378b0644509ed376022a1e6b59 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:45.448864 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a93a5-ca23-4caf-8774-7ad5b7228f0e service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:45.460013 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a93a5-ca23-4caf-8774-7ad5b7228f0e service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:45.517142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1cbc5-b9c0-46ad-a39d-201097df2e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:45.518034 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1cbc5-b9c0-46ad-a39d-201097df2e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:45.518316 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1cbc5-b9c0-46ad-a39d-201097df2e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:45.518624 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1cbc5-b9c0-46ad-a39d-201097df2e5e admin admin] PATH_INFO: `/v3/users/14e67fefe9844d64be9394ac18f65328` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:45.519849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1cbc5-b9c0-46ad-a39d-201097df2e5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=14e67fefe9844d64be9394ac18f65328)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:45.522355 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1cbc5-b9c0-46ad-a39d-201097df2e5e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:45.654717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6a3be-a352-49b9-af76-6a1021a5c361 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:45.656246 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-47c6a3be-a352-49b9-af76-6a1021a5c361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:45.723716 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6a3be-a352-49b9-af76-6a1021a5c361 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:45.725335 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-39c1cbc5-b9c0-46ad-a39d-201097df2e5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14e67fefe9844d64be9394ac18f65328'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:45.733078 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4ce93-83fd-423a-b8ef-0702033c8a32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:45.733997 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4ce93-83fd-423a-b8ef-0702033c8a32 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:45.734436 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4ce93-83fd-423a-b8ef-0702033c8a32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:45.734898 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4ce93-83fd-423a-b8ef-0702033c8a32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:45.736936 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-95c4ce93-83fd-423a-b8ef-0702033c8a32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:45.739715 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-39c1cbc5-b9c0-46ad-a39d-201097df2e5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14e67fefe9844d64be9394ac18f65328'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:45.741516 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-39c1cbc5-b9c0-46ad-a39d-201097df2e5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14e67fefe9844d64be9394ac18f65328'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:45.742378 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-39c1cbc5-b9c0-46ad-a39d-201097df2e5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14e67fefe9844d64be9394ac18f65328'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:45.757643 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:45.763492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-39c1cbc5-b9c0-46ad-a39d-201097df2e5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14e67fefe9844d64be9394ac18f65328'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:45.780393 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:45.815276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4ce93-83fd-423a-b8ef-0702033c8a32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:45.816894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4ce93-83fd-423a-b8ef-0702033c8a32 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:45.827402 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1038/2066] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:45 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:45.877521 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1029/2067] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:45 2022] DELETE /identity/v3/users/14e67fefe9844d64be9394ac18f65328 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:46.314664 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4ce93-83fd-423a-b8ef-0702033c8a32 service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:46.316130 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-95c4ce93-83fd-423a-b8ef-0702033c8a32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:46.332189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4ce93-83fd-423a-b8ef-0702033c8a32 service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:46.400633 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8318c9-cf41-4885-aa30-080895aa4033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:46.401700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8318c9-cf41-4885-aa30-080895aa4033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:46.402154 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8318c9-cf41-4885-aa30-080895aa4033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:46.402487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8318c9-cf41-4885-aa30-080895aa4033 admin admin] PATH_INFO: `/v3/projects/126d433f939d4bbd906494ab8dde9648` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:46.404344 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8318c9-cf41-4885-aa30-080895aa4033 admin admin] RBAC: Authorizing `identity:delete_project(project_id=126d433f939d4bbd906494ab8dde9648)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:46.407117 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8318c9-cf41-4885-aa30-080895aa4033 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:46.561548 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4f8318c9-cf41-4885-aa30-080895aa4033 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '126d433f939d4bbd906494ab8dde9648', 'request_id': 'req-4f8318c9-cf41-4885-aa30-080895aa4033'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:46.569174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4f8318c9-cf41-4885-aa30-080895aa4033 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '126d433f939d4bbd906494ab8dde9648', 'request_id': 'req-4f8318c9-cf41-4885-aa30-080895aa4033'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:46.648379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4f8318c9-cf41-4885-aa30-080895aa4033 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '126d433f939d4bbd906494ab8dde9648', 'request_id': 'req-4f8318c9-cf41-4885-aa30-080895aa4033'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:46.660319 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4f8318c9-cf41-4885-aa30-080895aa4033 admin admin] The token cache is being invalidate because project 126d433f939d4bbd906494ab8dde9648 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:46.660537 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4f8318c9-cf41-4885-aa30-080895aa4033 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:46.661434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4f8318c9-cf41-4885-aa30-080895aa4033 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:46.663412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1039/2068] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:46 2022] DELETE /identity/v3/projects/126d433f939d4bbd906494ab8dde9648 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:46.668544 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a0037-6c93-47e0-b7c9-0bb6caca6ec1 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:46.668999 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a0037-6c93-47e0-b7c9-0bb6caca6ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:46.669282 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a0037-6c93-47e0-b7c9-0bb6caca6ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:46.689413 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-bf4a0037-6c93-47e0-b7c9-0bb6caca6ec1 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:46.690672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bf4a0037-6c93-47e0-b7c9-0bb6caca6ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:46.758173 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1030/2069] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:46.766436 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205127cf-1d84-4775-8049-1f15e9e51a7e None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:46.766988 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205127cf-1d84-4775-8049-1f15e9e51a7e None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:46.767165 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205127cf-1d84-4775-8049-1f15e9e51a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:46.788069 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-205127cf-1d84-4775-8049-1f15e9e51a7e None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:46.789350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-205127cf-1d84-4775-8049-1f15e9e51a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:46.797779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1040/2070] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:45:46.802075 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a0037-6c93-47e0-b7c9-0bb6caca6ec1 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:46.814836 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a0037-6c93-47e0-b7c9-0bb6caca6ec1 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:46.821092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e30a7-49fe-4678-8203-f9354a19e30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:46.822007 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e30a7-49fe-4678-8203-f9354a19e30e admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:46.822290 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e30a7-49fe-4678-8203-f9354a19e30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:46.822853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e30a7-49fe-4678-8203-f9354a19e30e admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:46.823592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e30a7-49fe-4678-8203-f9354a19e30e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:46.824057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e30a7-49fe-4678-8203-f9354a19e30e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:46.825832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e30a7-49fe-4678-8203-f9354a19e30e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:46.841208 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1031/2071] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:46.850968 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205127cf-1d84-4775-8049-1f15e9e51a7e None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:46.866074 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205127cf-1d84-4775-8049-1f15e9e51a7e None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:46.872735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b33d2e-0e9a-410f-a0c5-52f957ec8f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:46.873829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b33d2e-0e9a-410f-a0c5-52f957ec8f05 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:46.874945 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b33d2e-0e9a-410f-a0c5-52f957ec8f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:46.882495 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b33d2e-0e9a-410f-a0c5-52f957ec8f05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:46.882495 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b33d2e-0e9a-410f-a0c5-52f957ec8f05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:46.882495 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b33d2e-0e9a-410f-a0c5-52f957ec8f05 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:46.972513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1041/2072] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:46 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:46.973842 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e30a7-49fe-4678-8203-f9354a19e30e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:46.986333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e30a7-49fe-4678-8203-f9354a19e30e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:47.057367 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d10d8-5dae-4255-b819-389acce465d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:47.058039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d10d8-5dae-4255-b819-389acce465d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:47.058329 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d10d8-5dae-4255-b819-389acce465d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:47.059111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d10d8-5dae-4255-b819-389acce465d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:47.060104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d10d8-5dae-4255-b819-389acce465d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:47.062516 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d10d8-5dae-4255-b819-389acce465d8 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:47.159954 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1032/2073] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:46 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:47.165358 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b33d2e-0e9a-410f-a0c5-52f957ec8f05 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:47.178982 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b33d2e-0e9a-410f-a0c5-52f957ec8f05 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:47.185011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bdc8d-5c19-4d56-89c3-dd05cdac5fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:47.185776 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bdc8d-5c19-4d56-89c3-dd05cdac5fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:47.186136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bdc8d-5c19-4d56-89c3-dd05cdac5fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:47.186477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bdc8d-5c19-4d56-89c3-dd05cdac5fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:47.187382 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45bdc8d-5c19-4d56-89c3-dd05cdac5fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:47.189250 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45bdc8d-5c19-4d56-89c3-dd05cdac5fe8 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:47.206147 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1042/2074] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:45:47.212993 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d10d8-5dae-4255-b819-389acce465d8 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:47.227062 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d10d8-5dae-4255-b819-389acce465d8 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:47.232902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a1c7d-6f25-408e-ab34-0c71bfe4cc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:47.233835 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a1c7d-6f25-408e-ab34-0c71bfe4cc57 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:47.234197 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a1c7d-6f25-408e-ab34-0c71bfe4cc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:47.234628 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a1c7d-6f25-408e-ab34-0c71bfe4cc57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:47.236129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6a1c7d-6f25-408e-ab34-0c71bfe4cc57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:47.238072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6a1c7d-6f25-408e-ab34-0c71bfe4cc57 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:47.253960 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1033/2075] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:47.256682 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bdc8d-5c19-4d56-89c3-dd05cdac5fe8 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:47.267999 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bdc8d-5c19-4d56-89c3-dd05cdac5fe8 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:47.273732 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa280e3-00a5-4217-8b4f-c45b14db81e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:47.274784 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa280e3-00a5-4217-8b4f-c45b14db81e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:47.279186 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa280e3-00a5-4217-8b4f-c45b14db81e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:47.283053 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa280e3-00a5-4217-8b4f-c45b14db81e5 admin admin] PATH_INFO: `/v3/projects/ce01d9a78d3f4bb7b101b47548b73037/users/626c3dbdc0f1407db82d6988e68e40b2/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:47.296934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa280e3-00a5-4217-8b4f-c45b14db81e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce01d9a78d3f4bb7b101b47548b73037, user_id=626c3dbdc0f1407db82d6988e68e40b2, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:47.299684 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa280e3-00a5-4217-8b4f-c45b14db81e5 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:47.303569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:47.373324 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1043/2076] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:47 2022] PUT /identity/v3/projects/ce01d9a78d3f4bb7b101b47548b73037/users/626c3dbdc0f1407db82d6988e68e40b2/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:47.378590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16296cde-1b64-4803-8f6c-cf1aa42559de None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:47.378823 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16296cde-1b64-4803-8f6c-cf1aa42559de None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:47.379108 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16296cde-1b64-4803-8f6c-cf1aa42559de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:47.399531 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-16296cde-1b64-4803-8f6c-cf1aa42559de None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:47.400755 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-16296cde-1b64-4803-8f6c-cf1aa42559de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:47.429930 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:47.461314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1034/2077] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:47.470809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa280e3-00a5-4217-8b4f-c45b14db81e5 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:47.472559 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-5fa280e3-00a5-4217-8b4f-c45b14db81e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:47.532357 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa280e3-00a5-4217-8b4f-c45b14db81e5 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:47.536086 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47242860-6e74-4b04-ae3e-02582d8330a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:47.536803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47242860-6e74-4b04-ae3e-02582d8330a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:47.537102 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47242860-6e74-4b04-ae3e-02582d8330a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:47.537378 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47242860-6e74-4b04-ae3e-02582d8330a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:47.549205 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47242860-6e74-4b04-ae3e-02582d8330a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:47.550766 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47242860-6e74-4b04-ae3e-02582d8330a1 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:47.562822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1044/2078] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:53.142203 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17428b9c-047d-405e-b9bb-9fc5d8ba0f66 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:53.142722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17428b9c-047d-405e-b9bb-9fc5d8ba0f66 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:53.142843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17428b9c-047d-405e-b9bb-9fc5d8ba0f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:53.172864 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-17428b9c-047d-405e-b9bb-9fc5d8ba0f66 None None] MFA Rules not processed for user `626c3dbdc0f1407db82d6988e68e40b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:53.174078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-17428b9c-047d-405e-b9bb-9fc5d8ba0f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:53.257493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1035/2079] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:45:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:53.261829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47242860-6e74-4b04-ae3e-02582d8330a1 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:53.272276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47242860-6e74-4b04-ae3e-02582d8330a1 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:53.276955 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdc578-2320-4541-bbf8-c59c456892c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:53.277615 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdc578-2320-4541-bbf8-c59c456892c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:53.277892 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdc578-2320-4541-bbf8-c59c456892c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:53.278165 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdc578-2320-4541-bbf8-c59c456892c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:53.278910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fdc578-2320-4541-bbf8-c59c456892c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:53.280838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fdc578-2320-4541-bbf8-c59c456892c6 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:53.381809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1045/2080] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:53 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:53.385890 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17428b9c-047d-405e-b9bb-9fc5d8ba0f66 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:53.398921 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17428b9c-047d-405e-b9bb-9fc5d8ba0f66 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:53.459718 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7839a3-aa86-4489-a40d-f34c472f8738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:53.460432 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7839a3-aa86-4489-a40d-f34c472f8738 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:53.460720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7839a3-aa86-4489-a40d-f34c472f8738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:53.460998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7839a3-aa86-4489-a40d-f34c472f8738 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:53.461915 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7839a3-aa86-4489-a40d-f34c472f8738 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:53.463975 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7839a3-aa86-4489-a40d-f34c472f8738 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:53.538041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1036/2081] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:53 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:53.542174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdc578-2320-4541-bbf8-c59c456892c6 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:53.553408 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdc578-2320-4541-bbf8-c59c456892c6 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:53.558799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00f3a0-b3e5-4715-a12f-09327908911c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:53.559477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00f3a0-b3e5-4715-a12f-09327908911c admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:53.559859 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00f3a0-b3e5-4715-a12f-09327908911c admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:53.560161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00f3a0-b3e5-4715-a12f-09327908911c admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:53.560838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00f3a0-b3e5-4715-a12f-09327908911c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:53.562848 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00f3a0-b3e5-4715-a12f-09327908911c admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:53.574801 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1046/2082] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:53.578922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7839a3-aa86-4489-a40d-f34c472f8738 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:53.589740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7839a3-aa86-4489-a40d-f34c472f8738 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:53.594439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee717d3-72be-4e07-a9ec-87a554e9e135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:53.595435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee717d3-72be-4e07-a9ec-87a554e9e135 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:53.595680 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee717d3-72be-4e07-a9ec-87a554e9e135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:53.595955 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee717d3-72be-4e07-a9ec-87a554e9e135 admin admin] PATH_INFO: `/v3/projects/28c51273c8554c95ab585206bf920f3e/users/c3a61b9ccab24c6882917e33812beead/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:53.616410 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee717d3-72be-4e07-a9ec-87a554e9e135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28c51273c8554c95ab585206bf920f3e, user_id=c3a61b9ccab24c6882917e33812beead, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:53.619096 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee717d3-72be-4e07-a9ec-87a554e9e135 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:53.716568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1037/2083] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:53 2022] PUT /identity/v3/projects/28c51273c8554c95ab585206bf920f3e/users/c3a61b9ccab24c6882917e33812beead/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:53.720510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00f3a0-b3e5-4715-a12f-09327908911c admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:53.730717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00f3a0-b3e5-4715-a12f-09327908911c admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:53.789516 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4b1df-8e23-4730-9b56-dc6c13cdbc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:53.790146 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4b1df-8e23-4730-9b56-dc6c13cdbc43 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:53.790460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4b1df-8e23-4730-9b56-dc6c13cdbc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:53.790767 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4b1df-8e23-4730-9b56-dc6c13cdbc43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:53.791507 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b4b1df-8e23-4730-9b56-dc6c13cdbc43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:53.793113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b4b1df-8e23-4730-9b56-dc6c13cdbc43 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:53.804326 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1047/2084] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:53.808475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee717d3-72be-4e07-a9ec-87a554e9e135 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:53.819992 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee717d3-72be-4e07-a9ec-87a554e9e135 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:53.824885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5ca46-1c8e-4f10-8abd-1a5a3fc915b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:53.825769 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5ca46-1c8e-4f10-8abd-1a5a3fc915b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:53.826051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5ca46-1c8e-4f10-8abd-1a5a3fc915b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:53.826330 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5ca46-1c8e-4f10-8abd-1a5a3fc915b2 admin admin] PATH_INFO: `/v3/projects/28c51273c8554c95ab585206bf920f3e/users/c3a61b9ccab24c6882917e33812beead/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:53.828330 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5ca46-1c8e-4f10-8abd-1a5a3fc915b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28c51273c8554c95ab585206bf920f3e, user_id=c3a61b9ccab24c6882917e33812beead, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:53.831058 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5ca46-1c8e-4f10-8abd-1a5a3fc915b2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:53.842175 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1038/2085] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:53 2022] PUT /identity/v3/projects/28c51273c8554c95ab585206bf920f3e/users/c3a61b9ccab24c6882917e33812beead/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:46:00.033793 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4b1df-8e23-4730-9b56-dc6c13cdbc43 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:00.036429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-05b4b1df-8e23-4730-9b56-dc6c13cdbc43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:00.051089 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4b1df-8e23-4730-9b56-dc6c13cdbc43 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:00.159407 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce7d67-58c5-4d2e-b59b-958c98ee6fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:00.159407 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce7d67-58c5-4d2e-b59b-958c98ee6fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:00.159407 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce7d67-58c5-4d2e-b59b-958c98ee6fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:00.159407 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce7d67-58c5-4d2e-b59b-958c98ee6fa1 admin admin] PATH_INFO: `/v3/users/f0cd4c9d85854f7f83aa56cbaaabd798` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:00.159765 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce7d67-58c5-4d2e-b59b-958c98ee6fa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0cd4c9d85854f7f83aa56cbaaabd798)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:00.159765 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce7d67-58c5-4d2e-b59b-958c98ee6fa1 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:00.162438 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:00.443838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-bcce7d67-58c5-4d2e-b59b-958c98ee6fa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0cd4c9d85854f7f83aa56cbaaabd798'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:00.453950 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-bcce7d67-58c5-4d2e-b59b-958c98ee6fa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0cd4c9d85854f7f83aa56cbaaabd798'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:00.455704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-bcce7d67-58c5-4d2e-b59b-958c98ee6fa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0cd4c9d85854f7f83aa56cbaaabd798'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:00.455912 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-bcce7d67-58c5-4d2e-b59b-958c98ee6fa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0cd4c9d85854f7f83aa56cbaaabd798'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:00.473085 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-bcce7d67-58c5-4d2e-b59b-958c98ee6fa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0cd4c9d85854f7f83aa56cbaaabd798'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:00.564942 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1048/2086] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:46:00 2022] DELETE /identity/v3/users/f0cd4c9d85854f7f83aa56cbaaabd798 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:00.568185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249c29b-130c-47fa-90d4-fb2f3a770af6 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:00.568681 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249c29b-130c-47fa-90d4-fb2f3a770af6 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:00.568820 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249c29b-130c-47fa-90d4-fb2f3a770af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:00.600859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5249c29b-130c-47fa-90d4-fb2f3a770af6 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:00.602014 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5249c29b-130c-47fa-90d4-fb2f3a770af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:00.615076 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:00.708194 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1039/2087] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:00.740846 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce7d67-58c5-4d2e-b59b-958c98ee6fa1 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:00.742290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-bcce7d67-58c5-4d2e-b59b-958c98ee6fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:00.932519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce7d67-58c5-4d2e-b59b-958c98ee6fa1 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:00.938084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e28a3d-b53a-4fe8-ac66-6fe604695f06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:00.938810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e28a3d-b53a-4fe8-ac66-6fe604695f06 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:00.939099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e28a3d-b53a-4fe8-ac66-6fe604695f06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:00.939566 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e28a3d-b53a-4fe8-ac66-6fe604695f06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:00.970502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e28a3d-b53a-4fe8-ac66-6fe604695f06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:00.972403 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e28a3d-b53a-4fe8-ac66-6fe604695f06 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:00.985220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1049/2088] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:01.750387 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfe4d6-e8b9-468e-b76a-264b74e4a8e4 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:01.750942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfe4d6-e8b9-468e-b76a-264b74e4a8e4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:01.751032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfe4d6-e8b9-468e-b76a-264b74e4a8e4 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:01.752974 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1040/2089] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 23 11:46:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:46:01.758822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db580ac5-b465-41dc-af00-ab1c994c2222 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:01.759833 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db580ac5-b465-41dc-af00-ab1c994c2222 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:01.760210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db580ac5-b465-41dc-af00-ab1c994c2222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:01.793146 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-db580ac5-b465-41dc-af00-ab1c994c2222 None None] MFA Rules not processed for user `ac04b32777ca4f929e9e59deee6e0260`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:01.795487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-db580ac5-b465-41dc-af00-ab1c994c2222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:01.890908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1050/2090] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 23 11:46:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:01.905321 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfe4d6-e8b9-468e-b76a-264b74e4a8e4 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:01.908610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-74bfe4d6-e8b9-468e-b76a-264b74e4a8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:01.939228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfe4d6-e8b9-468e-b76a-264b74e4a8e4 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:01.946344 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2dabe-e3cf-4cec-995e-3aade20a065d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac04b32777ca4f929e9e59deee6e0260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:01.947243 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2dabe-e3cf-4cec-995e-3aade20a065d service placement] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:01.947705 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2dabe-e3cf-4cec-995e-3aade20a065d service placement] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:01.948437 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2dabe-e3cf-4cec-995e-3aade20a065d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:01.962230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2dabe-e3cf-4cec-995e-3aade20a065d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:01.963759 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db580ac5-b465-41dc-af00-ab1c994c2222 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:01.964799 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2dabe-e3cf-4cec-995e-3aade20a065d service placement] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:01.968874 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-db580ac5-b465-41dc-af00-ab1c994c2222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:01.994396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db580ac5-b465-41dc-af00-ab1c994c2222 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:01.996821 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1041/2091] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:02.004478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b51280-540c-45c0-89d9-1bb5dfb34776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:02.005873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b51280-540c-45c0-89d9-1bb5dfb34776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:02.005873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b51280-540c-45c0-89d9-1bb5dfb34776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:02.006121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b51280-540c-45c0-89d9-1bb5dfb34776 admin admin] PATH_INFO: `/v3/projects/44105066239048dcb849ae61eb73c150` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:02.007771 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b51280-540c-45c0-89d9-1bb5dfb34776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44105066239048dcb849ae61eb73c150)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:02.011100 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b51280-540c-45c0-89d9-1bb5dfb34776 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:02.013223 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2dabe-e3cf-4cec-995e-3aade20a065d service placement] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:02.015784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-40c2dabe-e3cf-4cec-995e-3aade20a065d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:02.037495 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2dabe-e3cf-4cec-995e-3aade20a065d service placement] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:02.037864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:02.059355 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66700a45-5788-4376-acf5-ab58f8753a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:02.061111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66700a45-5788-4376-acf5-ab58f8753a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:02.061485 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66700a45-5788-4376-acf5-ab58f8753a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:02.061855 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66700a45-5788-4376-acf5-ab58f8753a99 admin admin] PATH_INFO: `/v3/users/ae72d6c494ba4f16989adfe090803d47` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:02.063359 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66700a45-5788-4376-acf5-ab58f8753a99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae72d6c494ba4f16989adfe090803d47)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:02.065841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66700a45-5788-4376-acf5-ab58f8753a99 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:02.133910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-89b51280-540c-45c0-89d9-1bb5dfb34776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44105066239048dcb849ae61eb73c150', 'request_id': 'req-89b51280-540c-45c0-89d9-1bb5dfb34776'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:02.141775 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-89b51280-540c-45c0-89d9-1bb5dfb34776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44105066239048dcb849ae61eb73c150', 'request_id': 'req-89b51280-540c-45c0-89d9-1bb5dfb34776'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:02.270427 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-89b51280-540c-45c0-89d9-1bb5dfb34776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44105066239048dcb849ae61eb73c150', 'request_id': 'req-89b51280-540c-45c0-89d9-1bb5dfb34776'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:02.300434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-89b51280-540c-45c0-89d9-1bb5dfb34776 admin admin] The token cache is being invalidate because project 44105066239048dcb849ae61eb73c150 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:46:02.301161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-89b51280-540c-45c0-89d9-1bb5dfb34776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:02.302874 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-89b51280-540c-45c0-89d9-1bb5dfb34776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:02.304791 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1051/2092] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:46:01 2022] DELETE /identity/v3/projects/44105066239048dcb849ae61eb73c150 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:46:02.309287 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b51280-540c-45c0-89d9-1bb5dfb34776 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:02.310176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-66700a45-5788-4376-acf5-ab58f8753a99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae72d6c494ba4f16989adfe090803d47'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:02.312285 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-89b51280-540c-45c0-89d9-1bb5dfb34776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:02.348644 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-66700a45-5788-4376-acf5-ab58f8753a99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae72d6c494ba4f16989adfe090803d47'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:02.353672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-66700a45-5788-4376-acf5-ab58f8753a99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae72d6c494ba4f16989adfe090803d47'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:02.354456 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-66700a45-5788-4376-acf5-ab58f8753a99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae72d6c494ba4f16989adfe090803d47'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:02.384923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-66700a45-5788-4376-acf5-ab58f8753a99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae72d6c494ba4f16989adfe090803d47'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:02.441228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b51280-540c-45c0-89d9-1bb5dfb34776 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:02.446935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cda75-23f8-439a-b7f7-5b0a045fa2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:02.447856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cda75-23f8-439a-b7f7-5b0a045fa2a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:02.448136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cda75-23f8-439a-b7f7-5b0a045fa2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:02.448408 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cda75-23f8-439a-b7f7-5b0a045fa2a5 admin admin] PATH_INFO: `/v3/users/e249d8a4289c4e3cb3cf9899169f8a48` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:02.449626 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cda75-23f8-439a-b7f7-5b0a045fa2a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e249d8a4289c4e3cb3cf9899169f8a48)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:02.451765 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cda75-23f8-439a-b7f7-5b0a045fa2a5 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:02.505649 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1042/2093] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:46:02 2022] DELETE /identity/v3/users/ae72d6c494ba4f16989adfe090803d47 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:46:02.507092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cffdc-4994-400f-a50e-db4ed0a99f2d None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:02.507382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cffdc-4994-400f-a50e-db4ed0a99f2d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:02.507655 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cffdc-4994-400f-a50e-db4ed0a99f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:02.542288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-229cffdc-4994-400f-a50e-db4ed0a99f2d None None] MFA Rules not processed for user `c3a61b9ccab24c6882917e33812beead`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:02.543878 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-229cffdc-4994-400f-a50e-db4ed0a99f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:02.666215 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-069cda75-23f8-439a-b7f7-5b0a045fa2a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e249d8a4289c4e3cb3cf9899169f8a48'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:02.668695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:02.677611 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1043/2094] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:02 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:02.680571 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-069cda75-23f8-439a-b7f7-5b0a045fa2a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e249d8a4289c4e3cb3cf9899169f8a48'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:02.685686 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-069cda75-23f8-439a-b7f7-5b0a045fa2a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e249d8a4289c4e3cb3cf9899169f8a48'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:02.686955 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-069cda75-23f8-439a-b7f7-5b0a045fa2a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e249d8a4289c4e3cb3cf9899169f8a48'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:02.691902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229cffdc-4994-400f-a50e-db4ed0a99f2d None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:02.697453 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-229cffdc-4994-400f-a50e-db4ed0a99f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:02.736827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-069cda75-23f8-439a-b7f7-5b0a045fa2a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e249d8a4289c4e3cb3cf9899169f8a48'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:02.812247 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229cffdc-4994-400f-a50e-db4ed0a99f2d None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:02.823530 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcd6cc-8a00-4a0a-b5c9-6c6f295c6591 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:02.824464 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcd6cc-8a00-4a0a-b5c9-6c6f295c6591 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:02.824880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcd6cc-8a00-4a0a-b5c9-6c6f295c6591 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:02.825300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcd6cc-8a00-4a0a-b5c9-6c6f295c6591 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:02.827601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-37dcd6cc-8a00-4a0a-b5c9-6c6f295c6591 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:02.858319 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1052/2095] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:46:02 2022] DELETE /identity/v3/users/e249d8a4289c4e3cb3cf9899169f8a48 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:46:02.931785 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dcd6cc-8a00-4a0a-b5c9-6c6f295c6591 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:02.933692 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dcd6cc-8a00-4a0a-b5c9-6c6f295c6591 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:02.945109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1044/2096] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:46:02.956803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cda75-23f8-439a-b7f7-5b0a045fa2a5 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:02.958695 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-069cda75-23f8-439a-b7f7-5b0a045fa2a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:03.023856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:03.066712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cda75-23f8-439a-b7f7-5b0a045fa2a5 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:03.071467 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932802e9-9e99-4841-aa62-2a674984c2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:03.072357 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932802e9-9e99-4841-aa62-2a674984c2f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:03.072646 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932802e9-9e99-4841-aa62-2a674984c2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:03.072920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932802e9-9e99-4841-aa62-2a674984c2f7 admin admin] PATH_INFO: `/v3/projects/515e65a705ee46cc8f8e03efbe58f177` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:03.074112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932802e9-9e99-4841-aa62-2a674984c2f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=515e65a705ee46cc8f8e03efbe58f177)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:03.076090 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932802e9-9e99-4841-aa62-2a674984c2f7 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:03.210395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-932802e9-9e99-4841-aa62-2a674984c2f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '515e65a705ee46cc8f8e03efbe58f177', 'request_id': 'req-932802e9-9e99-4841-aa62-2a674984c2f7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:03.216687 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-932802e9-9e99-4841-aa62-2a674984c2f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '515e65a705ee46cc8f8e03efbe58f177', 'request_id': 'req-932802e9-9e99-4841-aa62-2a674984c2f7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:03.344669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-932802e9-9e99-4841-aa62-2a674984c2f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '515e65a705ee46cc8f8e03efbe58f177', 'request_id': 'req-932802e9-9e99-4841-aa62-2a674984c2f7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:03.356584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-932802e9-9e99-4841-aa62-2a674984c2f7 admin admin] The token cache is being invalidate because project 515e65a705ee46cc8f8e03efbe58f177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:46:03.356870 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-932802e9-9e99-4841-aa62-2a674984c2f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:03.357484 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-932802e9-9e99-4841-aa62-2a674984c2f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:03.359059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1053/2097] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:46:02 2022] DELETE /identity/v3/projects/515e65a705ee46cc8f8e03efbe58f177 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:03.364386 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32717e7-e08a-451b-9cb6-15efb1ff1432 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:03.364688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32717e7-e08a-451b-9cb6-15efb1ff1432 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:03.365013 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32717e7-e08a-451b-9cb6-15efb1ff1432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:03.383314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-e32717e7-e08a-451b-9cb6-15efb1ff1432 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:03.384529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e32717e7-e08a-451b-9cb6-15efb1ff1432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:03.439949 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932802e9-9e99-4841-aa62-2a674984c2f7 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:03.441405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-932802e9-9e99-4841-aa62-2a674984c2f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:03.452500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1045/2098] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:03.454061 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6f2f1-88b3-42bb-8e6b-2b48a632ed63 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:03.455158 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6f2f1-88b3-42bb-8e6b-2b48a632ed63 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:03.455296 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6f2f1-88b3-42bb-8e6b-2b48a632ed63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:03.463861 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932802e9-9e99-4841-aa62-2a674984c2f7 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:03.469256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5302899-f227-437c-bdd0-a7a37e8f81e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:03.470116 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5302899-f227-437c-bdd0-a7a37e8f81e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:03.470395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5302899-f227-437c-bdd0-a7a37e8f81e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:03.470953 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5302899-f227-437c-bdd0-a7a37e8f81e0 admin admin] PATH_INFO: `/v3/projects/c04f546dbed548ad9c7b90782808c06b` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:03.472258 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5302899-f227-437c-bdd0-a7a37e8f81e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c04f546dbed548ad9c7b90782808c06b)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:03.475285 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5302899-f227-437c-bdd0-a7a37e8f81e0 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:03.478365 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-84a6f2f1-88b3-42bb-8e6b-2b48a632ed63 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:03.479729 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-84a6f2f1-88b3-42bb-8e6b-2b48a632ed63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:03.498835 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1046/2099] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:03.503544 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6f2f1-88b3-42bb-8e6b-2b48a632ed63 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:03.514502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6f2f1-88b3-42bb-8e6b-2b48a632ed63 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:03.520402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306912fe-4514-4655-aa35-990dba0076f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:03.521087 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306912fe-4514-4655-aa35-990dba0076f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:03.521376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306912fe-4514-4655-aa35-990dba0076f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:03.521893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306912fe-4514-4655-aa35-990dba0076f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:03.522393 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306912fe-4514-4655-aa35-990dba0076f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:46:03.523035 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306912fe-4514-4655-aa35-990dba0076f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:03.524880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306912fe-4514-4655-aa35-990dba0076f5 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:03.558346 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1047/2100] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:46:03.558346 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306912fe-4514-4655-aa35-990dba0076f5 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:03.558346 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306912fe-4514-4655-aa35-990dba0076f5 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:03.562947 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fecef-48be-4f76-a27c-1623f8039447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:03.563604 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fecef-48be-4f76-a27c-1623f8039447 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:03.563895 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fecef-48be-4f76-a27c-1623f8039447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:03.564172 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fecef-48be-4f76-a27c-1623f8039447 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:03.564915 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4fecef-48be-4f76-a27c-1623f8039447 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:03.566974 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4fecef-48be-4f76-a27c-1623f8039447 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:03.617041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-e5302899-f227-437c-bdd0-a7a37e8f81e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c04f546dbed548ad9c7b90782808c06b', 'request_id': 'req-e5302899-f227-437c-bdd0-a7a37e8f81e0'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:03.623309 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-e5302899-f227-437c-bdd0-a7a37e8f81e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c04f546dbed548ad9c7b90782808c06b', 'request_id': 'req-e5302899-f227-437c-bdd0-a7a37e8f81e0'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:03.636069 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:03.691445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1048/2101] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:46:03 2022] POST /identity/v3/projects => generated 389 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:46:03.693104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fecef-48be-4f76-a27c-1623f8039447 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:03.711852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-e5302899-f227-437c-bdd0-a7a37e8f81e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c04f546dbed548ad9c7b90782808c06b', 'request_id': 'req-e5302899-f227-437c-bdd0-a7a37e8f81e0'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:03.726588 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-e5302899-f227-437c-bdd0-a7a37e8f81e0 admin admin] The token cache is being invalidate because project c04f546dbed548ad9c7b90782808c06b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:46:03.726892 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-e5302899-f227-437c-bdd0-a7a37e8f81e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:03.727588 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-e5302899-f227-437c-bdd0-a7a37e8f81e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:03.732804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1054/2102] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:46:03 2022] DELETE /identity/v3/projects/c04f546dbed548ad9c7b90782808c06b => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:46:03.745295 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2da1f-5341-4253-a9d6-5116ef4a4f0b None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:03.758366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2da1f-5341-4253-a9d6-5116ef4a4f0b None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:03.758638 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2da1f-5341-4253-a9d6-5116ef4a4f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:03.792790 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-f0d2da1f-5341-4253-a9d6-5116ef4a4f0b None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:03.794013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f0d2da1f-5341-4253-a9d6-5116ef4a4f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:03.795978 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fecef-48be-4f76-a27c-1623f8039447 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:03.801518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47cfb5-a64e-4e26-bbc2-5fbaeb7a4d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:03.802347 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47cfb5-a64e-4e26-bbc2-5fbaeb7a4d78 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:03.802677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47cfb5-a64e-4e26-bbc2-5fbaeb7a4d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:03.802947 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47cfb5-a64e-4e26-bbc2-5fbaeb7a4d78 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:03.803737 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47cfb5-a64e-4e26-bbc2-5fbaeb7a4d78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:03.804258 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1055/2103] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:03.807545 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47cfb5-a64e-4e26-bbc2-5fbaeb7a4d78 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:03.813099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac43df-fcf6-4207-a2ca-8810b1ad15a6 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:03.817363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac43df-fcf6-4207-a2ca-8810b1ad15a6 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:03.817363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac43df-fcf6-4207-a2ca-8810b1ad15a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:03.847061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c1ac43df-fcf6-4207-a2ca-8810b1ad15a6 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:03.848399 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c1ac43df-fcf6-4207-a2ca-8810b1ad15a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:03.860818 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1056/2104] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:03.865837 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac43df-fcf6-4207-a2ca-8810b1ad15a6 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:03.876237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac43df-fcf6-4207-a2ca-8810b1ad15a6 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:03.882257 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982e4c6-3c56-47c2-8c09-824c0a2e1ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:03.883090 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982e4c6-3c56-47c2-8c09-824c0a2e1ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:03.883452 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982e4c6-3c56-47c2-8c09-824c0a2e1ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:03.883801 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982e4c6-3c56-47c2-8c09-824c0a2e1ee7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:03.884781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982e4c6-3c56-47c2-8c09-824c0a2e1ee7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:46:03.885108 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982e4c6-3c56-47c2-8c09-824c0a2e1ee7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:03.886878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982e4c6-3c56-47c2-8c09-824c0a2e1ee7 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:03.902720 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1057/2105] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:46:03.908170 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1049/2106] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:46:03 2022] POST /identity/v3/users => generated 295 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:46:03.908830 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982e4c6-3c56-47c2-8c09-824c0a2e1ee7 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:03.913103 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47cfb5-a64e-4e26-bbc2-5fbaeb7a4d78 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:03.915381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-fa47cfb5-a64e-4e26-bbc2-5fbaeb7a4d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:03.923181 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982e4c6-3c56-47c2-8c09-824c0a2e1ee7 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:03.929120 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61557c9b-d034-4347-b9c2-c0d97a0e45a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:03.932345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61557c9b-d034-4347-b9c2-c0d97a0e45a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:03.932345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61557c9b-d034-4347-b9c2-c0d97a0e45a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:03.932345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61557c9b-d034-4347-b9c2-c0d97a0e45a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:03.932568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47cfb5-a64e-4e26-bbc2-5fbaeb7a4d78 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:03.932670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61557c9b-d034-4347-b9c2-c0d97a0e45a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:03.933383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61557c9b-d034-4347-b9c2-c0d97a0e45a1 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:03.941600 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d78b79-fafe-4452-a240-b5217d51cf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:03.943024 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d78b79-fafe-4452-a240-b5217d51cf23 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:03.943468 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d78b79-fafe-4452-a240-b5217d51cf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:03.944005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d78b79-fafe-4452-a240-b5217d51cf23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:03.944838 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d78b79-fafe-4452-a240-b5217d51cf23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:03.946790 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d78b79-fafe-4452-a240-b5217d51cf23 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:03.972250 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1050/2107] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:46:03.974642 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d78b79-fafe-4452-a240-b5217d51cf23 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:03.995446 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d78b79-fafe-4452-a240-b5217d51cf23 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:04.000520 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1d54b-4da5-408d-a286-bd361c3e353c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:04.001944 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1d54b-4da5-408d-a286-bd361c3e353c admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:04.002235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1d54b-4da5-408d-a286-bd361c3e353c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:04.002515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1d54b-4da5-408d-a286-bd361c3e353c admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:04.003635 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1d54b-4da5-408d-a286-bd361c3e353c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:04.005697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1d54b-4da5-408d-a286-bd361c3e353c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:04.023040 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1051/2108] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:46:04.027245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1d54b-4da5-408d-a286-bd361c3e353c admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:04.040227 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1d54b-4da5-408d-a286-bd361c3e353c admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:04.045068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb1b83-0b39-4b58-903e-1a376c9b9326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:04.045959 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb1b83-0b39-4b58-903e-1a376c9b9326 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:04.050494 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb1b83-0b39-4b58-903e-1a376c9b9326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:04.050494 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb1b83-0b39-4b58-903e-1a376c9b9326 admin admin] PATH_INFO: `/v3/projects/4c0275f5304e4d9eae19c46a4f161217/users/41cb2b9a5156470aaeca9cc10f402ef9/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:04.071032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb1b83-0b39-4b58-903e-1a376c9b9326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c0275f5304e4d9eae19c46a4f161217, user_id=41cb2b9a5156470aaeca9cc10f402ef9, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:04.074028 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb1b83-0b39-4b58-903e-1a376c9b9326 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:04.085334 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1058/2109] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:46:03 2022] POST /identity/v3/projects => generated 367 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:46:04.086406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61557c9b-d034-4347-b9c2-c0d97a0e45a1 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:04.169485 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61557c9b-d034-4347-b9c2-c0d97a0e45a1 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:04.175147 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37754c5-d16e-4e97-9d03-0698034c8d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:04.175863 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37754c5-d16e-4e97-9d03-0698034c8d7a admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:04.176813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37754c5-d16e-4e97-9d03-0698034c8d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:04.177283 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37754c5-d16e-4e97-9d03-0698034c8d7a admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:04.178061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37754c5-d16e-4e97-9d03-0698034c8d7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:04.180077 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37754c5-d16e-4e97-9d03-0698034c8d7a admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:04.190890 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1052/2110] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:46:04 2022] PUT /identity/v3/projects/4c0275f5304e4d9eae19c46a4f161217/users/41cb2b9a5156470aaeca9cc10f402ef9/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:04.196412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb1b83-0b39-4b58-903e-1a376c9b9326 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:04.198322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c1bb1b83-0b39-4b58-903e-1a376c9b9326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:04.282182 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1059/2111] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:46:04 2022] POST /identity/v3/users => generated 284 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:46:04.283536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596c5d2-699d-4e90-a2e9-494f6b26837f None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:04.283817 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596c5d2-699d-4e90-a2e9-494f6b26837f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:04.284100 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596c5d2-699d-4e90-a2e9-494f6b26837f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:04.287538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb1b83-0b39-4b58-903e-1a376c9b9326 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:04.293577 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a96ca-518c-424c-a2d4-963f0f79a2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:04.294859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a96ca-518c-424c-a2d4-963f0f79a2ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:04.295296 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a96ca-518c-424c-a2d4-963f0f79a2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:04.295577 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a96ca-518c-424c-a2d4-963f0f79a2ac admin admin] PATH_INFO: `/v3/users/568a2758c088444ba34f1dcba8a981cb` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:04.297253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351a96ca-518c-424c-a2d4-963f0f79a2ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=568a2758c088444ba34f1dcba8a981cb)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:04.299735 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351a96ca-518c-424c-a2d4-963f0f79a2ac admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:04.303355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:04.310515 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-6596c5d2-699d-4e90-a2e9-494f6b26837f None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:04.312668 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-6596c5d2-699d-4e90-a2e9-494f6b26837f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:04.322497 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1060/2112] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:04.326459 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596c5d2-699d-4e90-a2e9-494f6b26837f None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:04.341979 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596c5d2-699d-4e90-a2e9-494f6b26837f None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:04.347108 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2effa32-0bc7-4f35-a499-3ccd126e1674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:04.347787 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2effa32-0bc7-4f35-a499-3ccd126e1674 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:04.348209 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2effa32-0bc7-4f35-a499-3ccd126e1674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:04.348585 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2effa32-0bc7-4f35-a499-3ccd126e1674 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:04.349621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2effa32-0bc7-4f35-a499-3ccd126e1674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:04.351307 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2effa32-0bc7-4f35-a499-3ccd126e1674 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:04.365805 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1061/2113] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:46:04.367490 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2effa32-0bc7-4f35-a499-3ccd126e1674 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:04.369688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e2effa32-0bc7-4f35-a499-3ccd126e1674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:04.453016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2effa32-0bc7-4f35-a499-3ccd126e1674 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:04.456787 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb2398-bde8-41f1-9a88-26c126f7b63c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:04.457432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb2398-bde8-41f1-9a88-26c126f7b63c service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:04.457727 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb2398-bde8-41f1-9a88-26c126f7b63c service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:04.458001 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb2398-bde8-41f1-9a88-26c126f7b63c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:04.468507 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb2398-bde8-41f1-9a88-26c126f7b63c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:04.470360 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb2398-bde8-41f1-9a88-26c126f7b63c service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:04.482059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1062/2114] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:04.483009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb2398-bde8-41f1-9a88-26c126f7b63c service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:04.501235 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb2398-bde8-41f1-9a88-26c126f7b63c service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:04.505571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-351a96ca-518c-424c-a2d4-963f0f79a2ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '568a2758c088444ba34f1dcba8a981cb'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:04.507635 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf712b9-bc09-48af-8927-8de588184d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:04.508867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf712b9-bc09-48af-8927-8de588184d96 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:04.509341 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf712b9-bc09-48af-8927-8de588184d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:04.509605 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf712b9-bc09-48af-8927-8de588184d96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:04.510734 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf712b9-bc09-48af-8927-8de588184d96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:04.512476 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf712b9-bc09-48af-8927-8de588184d96 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:04.514947 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-351a96ca-518c-424c-a2d4-963f0f79a2ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '568a2758c088444ba34f1dcba8a981cb'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:04.515762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-351a96ca-518c-424c-a2d4-963f0f79a2ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '568a2758c088444ba34f1dcba8a981cb'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:04.516429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-351a96ca-518c-424c-a2d4-963f0f79a2ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '568a2758c088444ba34f1dcba8a981cb'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:04.526103 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1063/2115] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:46:04.529752 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf712b9-bc09-48af-8927-8de588184d96 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:04.532538 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-7bf712b9-bc09-48af-8927-8de588184d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:04.543495 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-351a96ca-518c-424c-a2d4-963f0f79a2ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '568a2758c088444ba34f1dcba8a981cb'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:04.553951 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf712b9-bc09-48af-8927-8de588184d96 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:04.559810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae2ea8-1709-479f-ad04-852cc0c8d75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:04.564314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae2ea8-1709-479f-ad04-852cc0c8d75e admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:04.564314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae2ea8-1709-479f-ad04-852cc0c8d75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:04.564314 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae2ea8-1709-479f-ad04-852cc0c8d75e admin admin] PATH_INFO: `/v3/projects/c7e2aec290a640be8d25f379b153634f/users/27ac5f43b10e41d7809006bc9ecd21b7/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:04.581865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae2ea8-1709-479f-ad04-852cc0c8d75e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7e2aec290a640be8d25f379b153634f, user_id=27ac5f43b10e41d7809006bc9ecd21b7, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:04.585658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae2ea8-1709-479f-ad04-852cc0c8d75e admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:04.613420 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:04.652377 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1053/2116] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:46:04 2022] DELETE /identity/v3/users/568a2758c088444ba34f1dcba8a981cb => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:04.654404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66f2a9-1ff1-42b9-8e2b-64831168caeb None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:04.654822 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66f2a9-1ff1-42b9-8e2b-64831168caeb None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:04.655202 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66f2a9-1ff1-42b9-8e2b-64831168caeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:04.682714 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-cb66f2a9-1ff1-42b9-8e2b-64831168caeb None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:04.684351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-cb66f2a9-1ff1-42b9-8e2b-64831168caeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:04.690647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1064/2117] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:46:04 2022] PUT /identity/v3/projects/c7e2aec290a640be8d25f379b153634f/users/27ac5f43b10e41d7809006bc9ecd21b7/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:46:04.693681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a312ec0-cc0e-42f1-85ed-e05ab3193414 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:04.694117 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a312ec0-cc0e-42f1-85ed-e05ab3193414 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:04.694526 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a312ec0-cc0e-42f1-85ed-e05ab3193414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:04.738487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-7a312ec0-cc0e-42f1-85ed-e05ab3193414 None None] MFA Rules not processed for user `27ac5f43b10e41d7809006bc9ecd21b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:04.740452 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-7a312ec0-cc0e-42f1-85ed-e05ab3193414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:04.774681 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1054/2118] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:46:04.786163 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66f2a9-1ff1-42b9-8e2b-64831168caeb None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:04.788166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-cb66f2a9-1ff1-42b9-8e2b-64831168caeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:04.819919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1065/2119] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:04 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:46:04.824317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a312ec0-cc0e-42f1-85ed-e05ab3193414 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:04.838581 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a312ec0-cc0e-42f1-85ed-e05ab3193414 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:04.843586 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b6503-35f0-48a0-8bde-38902862e00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:04.844426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b6503-35f0-48a0-8bde-38902862e00c admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:04.844754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b6503-35f0-48a0-8bde-38902862e00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:04.845039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b6503-35f0-48a0-8bde-38902862e00c admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:04.859452 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b6503-35f0-48a0-8bde-38902862e00c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:04.863238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b6503-35f0-48a0-8bde-38902862e00c admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:04.871442 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66f2a9-1ff1-42b9-8e2b-64831168caeb None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:04.878464 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd566e1-fdd0-4fdc-81c4-bd021c4938df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:04.879464 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd566e1-fdd0-4fdc-81c4-bd021c4938df service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:04.879776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd566e1-fdd0-4fdc-81c4-bd021c4938df service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:04.880111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd566e1-fdd0-4fdc-81c4-bd021c4938df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:04.891620 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd566e1-fdd0-4fdc-81c4-bd021c4938df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:04.893585 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd566e1-fdd0-4fdc-81c4-bd021c4938df service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:04.905143 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1055/2120] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:05.019118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1066/2121] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:46:04 2022] POST /identity/v3/projects => generated 367 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:46:05.021817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd566e1-fdd0-4fdc-81c4-bd021c4938df service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:05.048092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd566e1-fdd0-4fdc-81c4-bd021c4938df service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:05.142633 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0eef4-cee3-482a-ac5b-ac998ee674c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:05.143395 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0eef4-cee3-482a-ac5b-ac998ee674c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:05.146117 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0eef4-cee3-482a-ac5b-ac998ee674c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:05.146117 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0eef4-cee3-482a-ac5b-ac998ee674c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:05.146117 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0eef4-cee3-482a-ac5b-ac998ee674c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:05.148797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0eef4-cee3-482a-ac5b-ac998ee674c0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:05.234581 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1056/2122] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:46:05 2022] POST /identity/v3/users => generated 290 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:46:05.235847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b6503-35f0-48a0-8bde-38902862e00c admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:05.253983 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b6503-35f0-48a0-8bde-38902862e00c admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:05.294964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f5086-3436-4dcc-92b4-3c9e31cfb11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:05.301089 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f5086-3436-4dcc-92b4-3c9e31cfb11b admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:05.303118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f5086-3436-4dcc-92b4-3c9e31cfb11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:05.303518 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f5086-3436-4dcc-92b4-3c9e31cfb11b admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:05.304386 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f5086-3436-4dcc-92b4-3c9e31cfb11b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:05.318151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f5086-3436-4dcc-92b4-3c9e31cfb11b admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:05.333203 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1067/2123] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:05 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:46:05.336944 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0eef4-cee3-482a-ac5b-ac998ee674c0 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:05.352144 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0eef4-cee3-482a-ac5b-ac998ee674c0 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:05.358331 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2fee4-d2c4-42e2-beeb-1e1ab2d3e4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:05.359350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2fee4-d2c4-42e2-beeb-1e1ab2d3e4ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:05.359764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2fee4-d2c4-42e2-beeb-1e1ab2d3e4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:05.360048 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2fee4-d2c4-42e2-beeb-1e1ab2d3e4ae admin admin] PATH_INFO: `/v3/projects/54897fffc8e14ed4965f8c605a2918d9/users/2ab32a2ddc254d9299ec807cf9faa615/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:05.382372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2fee4-d2c4-42e2-beeb-1e1ab2d3e4ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=54897fffc8e14ed4965f8c605a2918d9, user_id=2ab32a2ddc254d9299ec807cf9faa615, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:05.385856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2fee4-d2c4-42e2-beeb-1e1ab2d3e4ae admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:05.508677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1057/2124] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:46:05 2022] PUT /identity/v3/projects/54897fffc8e14ed4965f8c605a2918d9/users/2ab32a2ddc254d9299ec807cf9faa615/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:05.509995 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f5086-3436-4dcc-92b4-3c9e31cfb11b admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:05.522057 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f5086-3436-4dcc-92b4-3c9e31cfb11b admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:05.601815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d00076-1ee1-4d9e-b12f-8a40a8fdd9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:05.603211 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d00076-1ee1-4d9e-b12f-8a40a8fdd9b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:05.603509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d00076-1ee1-4d9e-b12f-8a40a8fdd9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:05.603792 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d00076-1ee1-4d9e-b12f-8a40a8fdd9b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:05.604660 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d00076-1ee1-4d9e-b12f-8a40a8fdd9b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:05.606345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d00076-1ee1-4d9e-b12f-8a40a8fdd9b5 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:05.619086 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1068/2125] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:05 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:46:05.623188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2fee4-d2c4-42e2-beeb-1e1ab2d3e4ae admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:05.635672 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d00076-1ee1-4d9e-b12f-8a40a8fdd9b5 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:05.637534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-83d00076-1ee1-4d9e-b12f-8a40a8fdd9b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:05.648172 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2fee4-d2c4-42e2-beeb-1e1ab2d3e4ae admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:05.652935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d00076-1ee1-4d9e-b12f-8a40a8fdd9b5 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:05.653985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a02f9-9102-41d4-935c-1b52f90a16b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:05.654952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a02f9-9102-41d4-935c-1b52f90a16b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:05.655420 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a02f9-9102-41d4-935c-1b52f90a16b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:05.655515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a02f9-9102-41d4-935c-1b52f90a16b1 admin admin] PATH_INFO: `/v3/projects/54897fffc8e14ed4965f8c605a2918d9/users/2ab32a2ddc254d9299ec807cf9faa615/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:05.659174 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a02f9-9102-41d4-935c-1b52f90a16b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54897fffc8e14ed4965f8c605a2918d9, user_id=2ab32a2ddc254d9299ec807cf9faa615, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:05.663125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a02f9-9102-41d4-935c-1b52f90a16b1 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:05.663540 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393780e-d45d-413e-bad9-18633d10e2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:05.664738 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393780e-d45d-413e-bad9-18633d10e2b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:05.665726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393780e-d45d-413e-bad9-18633d10e2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:05.666206 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393780e-d45d-413e-bad9-18633d10e2b7 admin admin] PATH_INFO: `/v3/projects/ae538de4a6f4486d84e8595d564872f1` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:05.668112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4393780e-d45d-413e-bad9-18633d10e2b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae538de4a6f4486d84e8595d564872f1)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:05.671449 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4393780e-d45d-413e-bad9-18633d10e2b7 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:05.678106 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1058/2126] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:46:05 2022] PUT /identity/v3/projects/54897fffc8e14ed4965f8c605a2918d9/users/2ab32a2ddc254d9299ec807cf9faa615/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:05.689715 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a6202-5d56-4f8e-b8f7-008b70782d57 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:05.690401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a6202-5d56-4f8e-b8f7-008b70782d57 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:05.690898 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a6202-5d56-4f8e-b8f7-008b70782d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:05.725438 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d32a6202-5d56-4f8e-b8f7-008b70782d57 None None] MFA Rules not processed for user `2ab32a2ddc254d9299ec807cf9faa615`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:05.727557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d32a6202-5d56-4f8e-b8f7-008b70782d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:05.828522 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4393780e-d45d-413e-bad9-18633d10e2b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae538de4a6f4486d84e8595d564872f1', 'request_id': 'req-4393780e-d45d-413e-bad9-18633d10e2b7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:05.835867 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1059/2127] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:05 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:46:05.844116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a6202-5d56-4f8e-b8f7-008b70782d57 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:05.845886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4393780e-d45d-413e-bad9-18633d10e2b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae538de4a6f4486d84e8595d564872f1', 'request_id': 'req-4393780e-d45d-413e-bad9-18633d10e2b7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:05.846080 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d32a6202-5d56-4f8e-b8f7-008b70782d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:05.888118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4393780e-d45d-413e-bad9-18633d10e2b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae538de4a6f4486d84e8595d564872f1', 'request_id': 'req-4393780e-d45d-413e-bad9-18633d10e2b7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:05.900311 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4393780e-d45d-413e-bad9-18633d10e2b7 admin admin] The token cache is being invalidate because project ae538de4a6f4486d84e8595d564872f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:46:05.900736 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4393780e-d45d-413e-bad9-18633d10e2b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:05.901533 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4393780e-d45d-413e-bad9-18633d10e2b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:05.908182 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1069/2128] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:46:05 2022] DELETE /identity/v3/projects/ae538de4a6f4486d84e8595d564872f1 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:05.909677 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdcedf-8647-4c40-acd1-4b59d14ff563 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:05.910108 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdcedf-8647-4c40-acd1-4b59d14ff563 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:05.911266 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdcedf-8647-4c40-acd1-4b59d14ff563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:05.912570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a6202-5d56-4f8e-b8f7-008b70782d57 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:05.931107 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-eefdcedf-8647-4c40-acd1-4b59d14ff563 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:05.932634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-eefdcedf-8647-4c40-acd1-4b59d14ff563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:05.947771 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:05.993456 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5e983-2eee-43d9-afd7-3d83af14b8fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:05.994273 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5e983-2eee-43d9-afd7-3d83af14b8fb service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:05.994783 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5e983-2eee-43d9-afd7-3d83af14b8fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:05.995276 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5e983-2eee-43d9-afd7-3d83af14b8fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:05.998043 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9dd5e983-2eee-43d9-afd7-3d83af14b8fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:06.028974 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1070/2129] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:06.045815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059059c-0b75-44d6-af2c-4058a8075110 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:06.045815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059059c-0b75-44d6-af2c-4058a8075110 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:06.045815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059059c-0b75-44d6-af2c-4058a8075110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:06.074732 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-d059059c-0b75-44d6-af2c-4058a8075110 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:06.074732 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d059059c-0b75-44d6-af2c-4058a8075110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:06.083448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1071/2130] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:06.088001 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059059c-0b75-44d6-af2c-4058a8075110 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:06.110012 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059059c-0b75-44d6-af2c-4058a8075110 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:06.111806 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5e983-2eee-43d9-afd7-3d83af14b8fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:06.114376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5e983-2eee-43d9-afd7-3d83af14b8fb service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:06.116035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ef491-e164-40f3-83a2-becbff25e5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:06.119319 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ef491-e164-40f3-83a2-becbff25e5cc admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:06.119838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ef491-e164-40f3-83a2-becbff25e5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:06.120442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ef491-e164-40f3-83a2-becbff25e5cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:06.122742 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ef491-e164-40f3-83a2-becbff25e5cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:46:06.123136 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:06.123256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ef491-e164-40f3-83a2-becbff25e5cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:06.127746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ef491-e164-40f3-83a2-becbff25e5cc admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:06.141056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1060/2131] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:05 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:06.143429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1072/2132] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:46:06.148363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5e983-2eee-43d9-afd7-3d83af14b8fb service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:06.161885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5e983-2eee-43d9-afd7-3d83af14b8fb service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:06.168046 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed9438-68fc-4aef-83e0-b4a62effb3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:06.168856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed9438-68fc-4aef-83e0-b4a62effb3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:06.169210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed9438-68fc-4aef-83e0-b4a62effb3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:06.169730 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed9438-68fc-4aef-83e0-b4a62effb3d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:06.170691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed9438-68fc-4aef-83e0-b4a62effb3d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:06.172691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed9438-68fc-4aef-83e0-b4a62effb3d4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:06.291143 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1061/2133] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:46:06 2022] POST /identity/v3/projects => generated 389 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:46:06.295406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ef491-e164-40f3-83a2-becbff25e5cc admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:06.315171 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ef491-e164-40f3-83a2-becbff25e5cc admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:06.406329 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed9438-68fc-4aef-83e0-b4a62effb3d4 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:06.408715 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-43ed9438-68fc-4aef-83e0-b4a62effb3d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:06.421345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3ebcb-b041-4674-8bdb-d46e43b26f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:06.428157 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3ebcb-b041-4674-8bdb-d46e43b26f76 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:06.428443 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3ebcb-b041-4674-8bdb-d46e43b26f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:06.428717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3ebcb-b041-4674-8bdb-d46e43b26f76 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:06.429461 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3ebcb-b041-4674-8bdb-d46e43b26f76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:06.431503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3ebcb-b041-4674-8bdb-d46e43b26f76 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:06.509398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed9438-68fc-4aef-83e0-b4a62effb3d4 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:06.522193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cca38b-7e2b-4dcf-815a-8fee8ba4c7f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:06.528947 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1073/2134] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:46:06 2022] POST /identity/v3/users => generated 295 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:46:06.532208 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cca38b-7e2b-4dcf-815a-8fee8ba4c7f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:06.532208 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cca38b-7e2b-4dcf-815a-8fee8ba4c7f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:06.532208 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cca38b-7e2b-4dcf-815a-8fee8ba4c7f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:06.545953 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3ebcb-b041-4674-8bdb-d46e43b26f76 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:06.546697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e9cca38b-7e2b-4dcf-815a-8fee8ba4c7f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:06.555092 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3ebcb-b041-4674-8bdb-d46e43b26f76 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:06.568213 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a847e-7f7a-4308-b313-7e3a4679223d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:06.569997 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a847e-7f7a-4308-b313-7e3a4679223d admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:06.571046 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a847e-7f7a-4308-b313-7e3a4679223d admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:06.571639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a847e-7f7a-4308-b313-7e3a4679223d admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:06.574327 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a847e-7f7a-4308-b313-7e3a4679223d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:06.579657 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a847e-7f7a-4308-b313-7e3a4679223d admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:06.615619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1074/2135] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:46:06.620854 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a847e-7f7a-4308-b313-7e3a4679223d admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:06.650766 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a847e-7f7a-4308-b313-7e3a4679223d admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:06.660081 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9821dd-4910-43d8-ab9c-1ed1db14471a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:06.660926 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9821dd-4910-43d8-ab9c-1ed1db14471a admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:06.661217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9821dd-4910-43d8-ab9c-1ed1db14471a admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:06.661832 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9821dd-4910-43d8-ab9c-1ed1db14471a admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:06.662698 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9821dd-4910-43d8-ab9c-1ed1db14471a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:06.664326 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9821dd-4910-43d8-ab9c-1ed1db14471a admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:06.680780 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1075/2136] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:46:06.685838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9821dd-4910-43d8-ab9c-1ed1db14471a admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:06.719639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9821dd-4910-43d8-ab9c-1ed1db14471a admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:06.720089 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cca38b-7e2b-4dcf-815a-8fee8ba4c7f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:06.721989 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cca38b-7e2b-4dcf-815a-8fee8ba4c7f1 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:06.727201 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4200e7-823d-4a53-b28a-4141b5459122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:06.729967 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4200e7-823d-4a53-b28a-4141b5459122 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:06.730644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4200e7-823d-4a53-b28a-4141b5459122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:06.731303 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4200e7-823d-4a53-b28a-4141b5459122 admin admin] PATH_INFO: `/v3/projects/c099614d2cf0413eae88fde46a1be0c8/users/6b9a202828a74bbdbb11973c5bf4bf96/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:06.736321 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1062/2137] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:06 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:06.750819 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4200e7-823d-4a53-b28a-4141b5459122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c099614d2cf0413eae88fde46a1be0c8, user_id=6b9a202828a74bbdbb11973c5bf4bf96, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:06.754273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4200e7-823d-4a53-b28a-4141b5459122 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:06.859488 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1076/2138] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:46:06 2022] PUT /identity/v3/projects/c099614d2cf0413eae88fde46a1be0c8/users/6b9a202828a74bbdbb11973c5bf4bf96/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:06.875345 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480f199-86bf-4aba-8549-4b016c3ac1f1 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:06.875737 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480f199-86bf-4aba-8549-4b016c3ac1f1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:06.879646 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480f199-86bf-4aba-8549-4b016c3ac1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:06.915493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8b3e1-41be-4796-8ecb-2b7bb0ef3e39 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:06.915493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8b3e1-41be-4796-8ecb-2b7bb0ef3e39 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:06.915493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8b3e1-41be-4796-8ecb-2b7bb0ef3e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:06.944483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3480f199-86bf-4aba-8549-4b016c3ac1f1 None None] MFA Rules not processed for user `6b9a202828a74bbdbb11973c5bf4bf96`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:06.945936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3480f199-86bf-4aba-8549-4b016c3ac1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:06.948556 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-0df8b3e1-41be-4796-8ecb-2b7bb0ef3e39 None None] MFA Rules not processed for user `626c3dbdc0f1407db82d6988e68e40b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:06.955459 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-0df8b3e1-41be-4796-8ecb-2b7bb0ef3e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:07.097721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1077/2139] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:07.105120 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1063/2140] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:46:07.127585 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:07.670413 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8b3e1-41be-4796-8ecb-2b7bb0ef3e39 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:07.844436 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8b3e1-41be-4796-8ecb-2b7bb0ef3e39 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:07.852395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2312118-3ef1-483b-9490-365ad5903064 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:07.855448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2312118-3ef1-483b-9490-365ad5903064 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:07.855929 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2312118-3ef1-483b-9490-365ad5903064 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:07.856268 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2312118-3ef1-483b-9490-365ad5903064 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:07.888363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2312118-3ef1-483b-9490-365ad5903064 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:07.889843 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2312118-3ef1-483b-9490-365ad5903064 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:07.915978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1078/2141] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:08.298851 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:09.491670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:09.659479 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:11.105457 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13595ac6-e900-4f36-baf5-f3525530beef None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:11.106043 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13595ac6-e900-4f36-baf5-f3525530beef None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:11.106043 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13595ac6-e900-4f36-baf5-f3525530beef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:11.135984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-13595ac6-e900-4f36-baf5-f3525530beef None None] MFA Rules not processed for user `41cb2b9a5156470aaeca9cc10f402ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:11.137238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-13595ac6-e900-4f36-baf5-f3525530beef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:11.214906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1064/2142] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:11.229476 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2312118-3ef1-483b-9490-365ad5903064 service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:11.231076 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c2312118-3ef1-483b-9490-365ad5903064 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:11.361519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2312118-3ef1-483b-9490-365ad5903064 service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:11.371410 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a2610f-f5e1-4d3e-90bd-a85ed8e7aa34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:11.371756 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2610f-f5e1-4d3e-90bd-a85ed8e7aa34 service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:11.372146 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2610f-f5e1-4d3e-90bd-a85ed8e7aa34 service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:11.372504 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2610f-f5e1-4d3e-90bd-a85ed8e7aa34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:11.384646 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a2610f-f5e1-4d3e-90bd-a85ed8e7aa34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:11.386509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a2610f-f5e1-4d3e-90bd-a85ed8e7aa34 service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:11.396437 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1079/2143] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:14.031853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affbfdf-2a7f-45a9-aaea-8d8be5d27cc2 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:14.032295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affbfdf-2a7f-45a9-aaea-8d8be5d27cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:14.032418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affbfdf-2a7f-45a9-aaea-8d8be5d27cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:14.051817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3affbfdf-2a7f-45a9-aaea-8d8be5d27cc2 None None] MFA Rules not processed for user `41cb2b9a5156470aaeca9cc10f402ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:14.053020 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3affbfdf-2a7f-45a9-aaea-8d8be5d27cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:14.060158 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1065/2144] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:20.200420 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a2610f-f5e1-4d3e-90bd-a85ed8e7aa34 service nova] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:20.202341 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d8a2610f-f5e1-4d3e-90bd-a85ed8e7aa34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:20.276875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a2610f-f5e1-4d3e-90bd-a85ed8e7aa34 service nova] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:20.281516 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533545b6-fa23-4f7d-9dfc-6a32152b6916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:20.282389 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533545b6-fa23-4f7d-9dfc-6a32152b6916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:20.282700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533545b6-fa23-4f7d-9dfc-6a32152b6916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:20.282980 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533545b6-fa23-4f7d-9dfc-6a32152b6916 admin admin] PATH_INFO: `/v3/users/27ac5f43b10e41d7809006bc9ecd21b7` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:20.284386 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533545b6-fa23-4f7d-9dfc-6a32152b6916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27ac5f43b10e41d7809006bc9ecd21b7)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:20.286973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533545b6-fa23-4f7d-9dfc-6a32152b6916 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:20.288744 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:20.421229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-533545b6-fa23-4f7d-9dfc-6a32152b6916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27ac5f43b10e41d7809006bc9ecd21b7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:20.430701 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-533545b6-fa23-4f7d-9dfc-6a32152b6916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27ac5f43b10e41d7809006bc9ecd21b7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:20.431438 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-533545b6-fa23-4f7d-9dfc-6a32152b6916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27ac5f43b10e41d7809006bc9ecd21b7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:20.432038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-533545b6-fa23-4f7d-9dfc-6a32152b6916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27ac5f43b10e41d7809006bc9ecd21b7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:20.444305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-533545b6-fa23-4f7d-9dfc-6a32152b6916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27ac5f43b10e41d7809006bc9ecd21b7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:20.509648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1080/2145] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:46:20 2022] DELETE /identity/v3/users/27ac5f43b10e41d7809006bc9ecd21b7 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:20.514263 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed52a54-5eba-4593-a13d-6f21b4426f26 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:20.514776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed52a54-5eba-4593-a13d-6f21b4426f26 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:20.514869 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed52a54-5eba-4593-a13d-6f21b4426f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:20.533292 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-4ed52a54-5eba-4593-a13d-6f21b4426f26 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:20.534478 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4ed52a54-5eba-4593-a13d-6f21b4426f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:20.604366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1066/2146] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:46:20.614148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533545b6-fa23-4f7d-9dfc-6a32152b6916 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:20.615904 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-533545b6-fa23-4f7d-9dfc-6a32152b6916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:20.678598 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533545b6-fa23-4f7d-9dfc-6a32152b6916 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:20.682059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea9437-9b3e-47a2-83b2-2a5c146d2c46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:20.682727 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea9437-9b3e-47a2-83b2-2a5c146d2c46 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:20.683009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea9437-9b3e-47a2-83b2-2a5c146d2c46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:20.683284 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea9437-9b3e-47a2-83b2-2a5c146d2c46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:20.727794 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea9437-9b3e-47a2-83b2-2a5c146d2c46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:20.729313 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea9437-9b3e-47a2-83b2-2a5c146d2c46 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:20.739585 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1081/2147] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:21.176415 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed52a54-5eba-4593-a13d-6f21b4426f26 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:21.178505 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4ed52a54-5eba-4593-a13d-6f21b4426f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:21.190817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed52a54-5eba-4593-a13d-6f21b4426f26 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:21.198397 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef716ee-eb64-4630-82ed-ccac8f0f80d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:21.198397 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef716ee-eb64-4630-82ed-ccac8f0f80d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:21.198397 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef716ee-eb64-4630-82ed-ccac8f0f80d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:21.198397 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef716ee-eb64-4630-82ed-ccac8f0f80d1 admin admin] PATH_INFO: `/v3/projects/c7e2aec290a640be8d25f379b153634f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:21.200384 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef716ee-eb64-4630-82ed-ccac8f0f80d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7e2aec290a640be8d25f379b153634f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:21.202063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef716ee-eb64-4630-82ed-ccac8f0f80d1 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:21.342202 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:21.345152 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2ef716ee-eb64-4630-82ed-ccac8f0f80d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7e2aec290a640be8d25f379b153634f', 'request_id': 'req-2ef716ee-eb64-4630-82ed-ccac8f0f80d1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:21.351748 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2ef716ee-eb64-4630-82ed-ccac8f0f80d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7e2aec290a640be8d25f379b153634f', 'request_id': 'req-2ef716ee-eb64-4630-82ed-ccac8f0f80d1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:21.434639 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2ef716ee-eb64-4630-82ed-ccac8f0f80d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7e2aec290a640be8d25f379b153634f', 'request_id': 'req-2ef716ee-eb64-4630-82ed-ccac8f0f80d1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:21.446385 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2ef716ee-eb64-4630-82ed-ccac8f0f80d1 admin admin] The token cache is being invalidate because project c7e2aec290a640be8d25f379b153634f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:46:21.446708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2ef716ee-eb64-4630-82ed-ccac8f0f80d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:21.447369 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2ef716ee-eb64-4630-82ed-ccac8f0f80d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:21.449059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1067/2148] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:46:21 2022] DELETE /identity/v3/projects/c7e2aec290a640be8d25f379b153634f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:21.453086 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea9437-9b3e-47a2-83b2-2a5c146d2c46 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:21.463385 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea9437-9b3e-47a2-83b2-2a5c146d2c46 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:21.521012 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87179df-9e79-45a1-9b68-73c5ee7fca78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:21.524549 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87179df-9e79-45a1-9b68-73c5ee7fca78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:21.524549 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87179df-9e79-45a1-9b68-73c5ee7fca78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:21.524549 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87179df-9e79-45a1-9b68-73c5ee7fca78 admin admin] PATH_INFO: `/v3/users/2ab32a2ddc254d9299ec807cf9faa615` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:21.524549 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87179df-9e79-45a1-9b68-73c5ee7fca78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ab32a2ddc254d9299ec807cf9faa615)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:21.525540 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87179df-9e79-45a1-9b68-73c5ee7fca78 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:21.547172 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:21.700362 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d87179df-9e79-45a1-9b68-73c5ee7fca78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ab32a2ddc254d9299ec807cf9faa615'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:21.706885 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d87179df-9e79-45a1-9b68-73c5ee7fca78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ab32a2ddc254d9299ec807cf9faa615'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:21.707687 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d87179df-9e79-45a1-9b68-73c5ee7fca78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ab32a2ddc254d9299ec807cf9faa615'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:21.708810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d87179df-9e79-45a1-9b68-73c5ee7fca78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ab32a2ddc254d9299ec807cf9faa615'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:21.720972 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d87179df-9e79-45a1-9b68-73c5ee7fca78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ab32a2ddc254d9299ec807cf9faa615'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:21.790241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1082/2149] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:46:21 2022] DELETE /identity/v3/users/2ab32a2ddc254d9299ec807cf9faa615 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:22.225978 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef716ee-eb64-4630-82ed-ccac8f0f80d1 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:22.227271 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2ef716ee-eb64-4630-82ed-ccac8f0f80d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:22.240630 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef716ee-eb64-4630-82ed-ccac8f0f80d1 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:22.310567 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dab74f-4853-4461-8280-bcfd69b782c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:22.311450 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dab74f-4853-4461-8280-bcfd69b782c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:22.311741 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dab74f-4853-4461-8280-bcfd69b782c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:22.312018 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dab74f-4853-4461-8280-bcfd69b782c9 admin admin] PATH_INFO: `/v3/projects/54897fffc8e14ed4965f8c605a2918d9` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:22.313231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dab74f-4853-4461-8280-bcfd69b782c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54897fffc8e14ed4965f8c605a2918d9)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:22.315197 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dab74f-4853-4461-8280-bcfd69b782c9 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:22.413360 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e2dab74f-4853-4461-8280-bcfd69b782c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54897fffc8e14ed4965f8c605a2918d9', 'request_id': 'req-e2dab74f-4853-4461-8280-bcfd69b782c9'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:22.420110 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e2dab74f-4853-4461-8280-bcfd69b782c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54897fffc8e14ed4965f8c605a2918d9', 'request_id': 'req-e2dab74f-4853-4461-8280-bcfd69b782c9'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:22.503530 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e2dab74f-4853-4461-8280-bcfd69b782c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54897fffc8e14ed4965f8c605a2918d9', 'request_id': 'req-e2dab74f-4853-4461-8280-bcfd69b782c9'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:22.515398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e2dab74f-4853-4461-8280-bcfd69b782c9 admin admin] The token cache is being invalidate because project 54897fffc8e14ed4965f8c605a2918d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:46:22.515686 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e2dab74f-4853-4461-8280-bcfd69b782c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:22.516336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e2dab74f-4853-4461-8280-bcfd69b782c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:22.517885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1068/2150] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:46:22 2022] DELETE /identity/v3/projects/54897fffc8e14ed4965f8c605a2918d9 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:22.523749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f719d-1388-4b5c-afcb-02680ace594d None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:22.524199 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f719d-1388-4b5c-afcb-02680ace594d None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:22.524287 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f719d-1388-4b5c-afcb-02680ace594d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:22.543345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-184f719d-1388-4b5c-afcb-02680ace594d None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:22.544558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-184f719d-1388-4b5c-afcb-02680ace594d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:22.605933 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1083/2151] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:22.614868 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b5f3c-b14a-4cb7-bdd3-8c6c6c7d2e0e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:22.615166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b5f3c-b14a-4cb7-bdd3-8c6c6c7d2e0e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:22.615444 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b5f3c-b14a-4cb7-bdd3-8c6c6c7d2e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:22.635360 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-804b5f3c-b14a-4cb7-bdd3-8c6c6c7d2e0e None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:22.636552 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-804b5f3c-b14a-4cb7-bdd3-8c6c6c7d2e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:22.646627 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1069/2152] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:22.650906 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f719d-1388-4b5c-afcb-02680ace594d None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:22.660584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f719d-1388-4b5c-afcb-02680ace594d None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:22.665722 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32afdd9-4a2b-4c08-b099-9e591f076f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:22.666393 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32afdd9-4a2b-4c08-b099-9e591f076f41 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:22.666717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32afdd9-4a2b-4c08-b099-9e591f076f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:22.666993 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32afdd9-4a2b-4c08-b099-9e591f076f41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:22.667819 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32afdd9-4a2b-4c08-b099-9e591f076f41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:46:22.668083 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32afdd9-4a2b-4c08-b099-9e591f076f41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:22.669688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32afdd9-4a2b-4c08-b099-9e591f076f41 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:22.681890 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1084/2153] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:46:22.686579 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b5f3c-b14a-4cb7-bdd3-8c6c6c7d2e0e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:22.697399 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b5f3c-b14a-4cb7-bdd3-8c6c6c7d2e0e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:22.702490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872c7a3-9039-44bc-afac-c4b6c4afe0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:22.703182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872c7a3-9039-44bc-afac-c4b6c4afe0fa admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:22.703478 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872c7a3-9039-44bc-afac-c4b6c4afe0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:22.703744 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872c7a3-9039-44bc-afac-c4b6c4afe0fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:22.704444 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9872c7a3-9039-44bc-afac-c4b6c4afe0fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:22.706388 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9872c7a3-9039-44bc-afac-c4b6c4afe0fa admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:22.784848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1070/2154] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:46:22 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:46:22.789230 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32afdd9-4a2b-4c08-b099-9e591f076f41 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:22.799304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32afdd9-4a2b-4c08-b099-9e591f076f41 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:22.857883 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a8fb0-8d97-4b00-9b41-312ebe839da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:22.858573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a8fb0-8d97-4b00-9b41-312ebe839da1 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:22.858857 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a8fb0-8d97-4b00-9b41-312ebe839da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:22.859132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a8fb0-8d97-4b00-9b41-312ebe839da1 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:22.859820 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a8fb0-8d97-4b00-9b41-312ebe839da1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:22.861791 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a8fb0-8d97-4b00-9b41-312ebe839da1 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:22.915012 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1085/2155] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:46:22 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:46:22.919406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872c7a3-9039-44bc-afac-c4b6c4afe0fa admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:22.930822 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872c7a3-9039-44bc-afac-c4b6c4afe0fa admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:22.935564 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057624c-f75c-443c-b881-d350f9b332ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:22.936230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057624c-f75c-443c-b881-d350f9b332ec admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:22.936519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057624c-f75c-443c-b881-d350f9b332ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:22.936792 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057624c-f75c-443c-b881-d350f9b332ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:22.937529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057624c-f75c-443c-b881-d350f9b332ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:22.939163 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057624c-f75c-443c-b881-d350f9b332ec admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:22.950897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1071/2156] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:46:22.954712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a8fb0-8d97-4b00-9b41-312ebe839da1 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:22.965012 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a8fb0-8d97-4b00-9b41-312ebe839da1 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:22.969548 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe49918-f6be-44d4-bd3d-c01bd96e375e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:22.970278 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe49918-f6be-44d4-bd3d-c01bd96e375e admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:22.970588 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe49918-f6be-44d4-bd3d-c01bd96e375e admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:22.970864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe49918-f6be-44d4-bd3d-c01bd96e375e admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:22.971581 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe49918-f6be-44d4-bd3d-c01bd96e375e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:22.973175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe49918-f6be-44d4-bd3d-c01bd96e375e admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:22.983439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1086/2157] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:46:22.987492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057624c-f75c-443c-b881-d350f9b332ec admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:22.998519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057624c-f75c-443c-b881-d350f9b332ec admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:23.003232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943383f-c867-465a-bbff-c74187951b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:23.004179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943383f-c867-465a-bbff-c74187951b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:23.004465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943383f-c867-465a-bbff-c74187951b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:23.005447 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943383f-c867-465a-bbff-c74187951b9c admin admin] PATH_INFO: `/v3/projects/6a51336b0b3f47468fe32d36e6b282ee/users/036fcf3b2b5d4223a44765dffc92a661/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:23.022134 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943383f-c867-465a-bbff-c74187951b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a51336b0b3f47468fe32d36e6b282ee, user_id=036fcf3b2b5d4223a44765dffc92a661, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:23.023495 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943383f-c867-465a-bbff-c74187951b9c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:23.082019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:23.087588 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1072/2158] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:46:22 2022] PUT /identity/v3/projects/6a51336b0b3f47468fe32d36e6b282ee/users/036fcf3b2b5d4223a44765dffc92a661/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:23.097674 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527d8d9-d82a-49ab-8003-45407a04c587 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:23.097969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527d8d9-d82a-49ab-8003-45407a04c587 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:23.098315 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527d8d9-d82a-49ab-8003-45407a04c587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:23.110668 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:23.124702 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-d527d8d9-d82a-49ab-8003-45407a04c587 None None] MFA Rules not processed for user `036fcf3b2b5d4223a44765dffc92a661`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:23.125907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d527d8d9-d82a-49ab-8003-45407a04c587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:23.196473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1087/2159] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:23.200920 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943383f-c867-465a-bbff-c74187951b9c admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:23.264442 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943383f-c867-465a-bbff-c74187951b9c admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:23.269116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478fa56-35d1-422d-a18f-21043f9c0028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:23.269775 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478fa56-35d1-422d-a18f-21043f9c0028 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:23.270063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478fa56-35d1-422d-a18f-21043f9c0028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:23.270391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478fa56-35d1-422d-a18f-21043f9c0028 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:23.271066 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478fa56-35d1-422d-a18f-21043f9c0028 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:23.272984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478fa56-35d1-422d-a18f-21043f9c0028 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:23.345121 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1073/2160] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:46:23 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:46:23.353097 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527d8d9-d82a-49ab-8003-45407a04c587 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:23.362012 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527d8d9-d82a-49ab-8003-45407a04c587 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:23.419201 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05b3e8-3cf4-4ca8-938d-80b3167dd6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:23.419873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05b3e8-3cf4-4ca8-938d-80b3167dd6b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:23.420159 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05b3e8-3cf4-4ca8-938d-80b3167dd6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:23.420443 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05b3e8-3cf4-4ca8-938d-80b3167dd6b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:23.421147 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05b3e8-3cf4-4ca8-938d-80b3167dd6b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:23.423118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05b3e8-3cf4-4ca8-938d-80b3167dd6b2 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:23.495124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1088/2161] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:46:23 2022] POST /identity/v3/users => generated 298 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:46:23.499199 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478fa56-35d1-422d-a18f-21043f9c0028 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:23.511080 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478fa56-35d1-422d-a18f-21043f9c0028 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:23.516349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b779e-db31-4f06-bb27-a3b619d8f4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:23.517131 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b779e-db31-4f06-bb27-a3b619d8f4ea admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:23.517488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b779e-db31-4f06-bb27-a3b619d8f4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:23.517839 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b779e-db31-4f06-bb27-a3b619d8f4ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:23.518680 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b779e-db31-4f06-bb27-a3b619d8f4ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:23.520503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b779e-db31-4f06-bb27-a3b619d8f4ea admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:23.534731 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1074/2162] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:46:23.535696 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05b3e8-3cf4-4ca8-938d-80b3167dd6b2 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:23.546260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05b3e8-3cf4-4ca8-938d-80b3167dd6b2 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:23.550896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820d2e2-8fd0-43f3-b256-309622bcf6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:23.552589 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820d2e2-8fd0-43f3-b256-309622bcf6a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:23.552868 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820d2e2-8fd0-43f3-b256-309622bcf6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:23.553140 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820d2e2-8fd0-43f3-b256-309622bcf6a7 admin admin] PATH_INFO: `/v3/projects/fa1e73231aca44778c023d295a8f61c1/users/0c27a222470f43a4b91d8e8f9d41db38/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:23.568478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820d2e2-8fd0-43f3-b256-309622bcf6a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa1e73231aca44778c023d295a8f61c1, user_id=0c27a222470f43a4b91d8e8f9d41db38, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:23.571115 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820d2e2-8fd0-43f3-b256-309622bcf6a7 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:23.635436 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1089/2163] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:46:23 2022] PUT /identity/v3/projects/fa1e73231aca44778c023d295a8f61c1/users/0c27a222470f43a4b91d8e8f9d41db38/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:23.639348 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b779e-db31-4f06-bb27-a3b619d8f4ea admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:23.650868 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b779e-db31-4f06-bb27-a3b619d8f4ea admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:23.708367 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653006b2-ce4d-430c-baa9-3cf68a28370e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:23.709353 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653006b2-ce4d-430c-baa9-3cf68a28370e admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:23.709633 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653006b2-ce4d-430c-baa9-3cf68a28370e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:23.709910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653006b2-ce4d-430c-baa9-3cf68a28370e admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:23.710661 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653006b2-ce4d-430c-baa9-3cf68a28370e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:23.712289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653006b2-ce4d-430c-baa9-3cf68a28370e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:23.722114 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1075/2164] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:46:23.726937 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820d2e2-8fd0-43f3-b256-309622bcf6a7 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:23.737355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820d2e2-8fd0-43f3-b256-309622bcf6a7 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:23.742028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026257d3-b92c-4465-abd4-3b367c79eb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:23.747390 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026257d3-b92c-4465-abd4-3b367c79eb49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:23.747390 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026257d3-b92c-4465-abd4-3b367c79eb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:23.747390 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026257d3-b92c-4465-abd4-3b367c79eb49 admin admin] PATH_INFO: `/v3/projects/fa1e73231aca44778c023d295a8f61c1/users/0c27a222470f43a4b91d8e8f9d41db38/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:23.747390 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026257d3-b92c-4465-abd4-3b367c79eb49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa1e73231aca44778c023d295a8f61c1, user_id=0c27a222470f43a4b91d8e8f9d41db38, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:23.748065 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026257d3-b92c-4465-abd4-3b367c79eb49 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:23.758775 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1090/2165] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:46:23 2022] PUT /identity/v3/projects/fa1e73231aca44778c023d295a8f61c1/users/0c27a222470f43a4b91d8e8f9d41db38/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:23.777815 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846b71a-3ce3-45e2-ba85-e1ecbf9686b3 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:23.778104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846b71a-3ce3-45e2-ba85-e1ecbf9686b3 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:23.778378 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846b71a-3ce3-45e2-ba85-e1ecbf9686b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:23.807223 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5846b71a-3ce3-45e2-ba85-e1ecbf9686b3 None None] MFA Rules not processed for user `0c27a222470f43a4b91d8e8f9d41db38`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:23.808601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5846b71a-3ce3-45e2-ba85-e1ecbf9686b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:23.869996 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:23.879245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1076/2166] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:23.890146 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026257d3-b92c-4465-abd4-3b367c79eb49 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:23.892017 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-026257d3-b92c-4465-abd4-3b367c79eb49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:23.964108 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:23.968126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026257d3-b92c-4465-abd4-3b367c79eb49 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:23.971816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164c515-80e3-4af8-a177-50d19f89f13a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:23.972466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164c515-80e3-4af8-a177-50d19f89f13a service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:23.972751 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164c515-80e3-4af8-a177-50d19f89f13a service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:23.973031 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164c515-80e3-4af8-a177-50d19f89f13a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:24.031242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164c515-80e3-4af8-a177-50d19f89f13a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:24.032765 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164c515-80e3-4af8-a177-50d19f89f13a service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:24.042060 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1091/2167] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:33.963552 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:34.212846 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=26 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:36.748637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846b71a-3ce3-45e2-ba85-e1ecbf9686b3 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:36.750080 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5846b71a-3ce3-45e2-ba85-e1ecbf9686b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:36.792262 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846b71a-3ce3-45e2-ba85-e1ecbf9686b3 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:36.795874 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585271c-30c6-422f-a015-8d5a29deb580 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:36.796499 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585271c-30c6-422f-a015-8d5a29deb580 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:36.796781 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585271c-30c6-422f-a015-8d5a29deb580 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:36.797055 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585271c-30c6-422f-a015-8d5a29deb580 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:36.826930 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b585271c-30c6-422f-a015-8d5a29deb580 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:36.828594 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b585271c-30c6-422f-a015-8d5a29deb580 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:36.838209 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1077/2168] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:39.797338 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164c515-80e3-4af8-a177-50d19f89f13a service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:39.798830 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-2164c515-80e3-4af8-a177-50d19f89f13a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:39.940190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164c515-80e3-4af8-a177-50d19f89f13a service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:39.944954 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666e79d-a9fa-4d4f-b4b1-f22292bd1294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:39.946091 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666e79d-a9fa-4d4f-b4b1-f22292bd1294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:39.946377 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666e79d-a9fa-4d4f-b4b1-f22292bd1294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:39.946686 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666e79d-a9fa-4d4f-b4b1-f22292bd1294 admin admin] PATH_INFO: `/v3/users/fc5b5e637c9849fcb05a669133eef001` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:39.947876 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666e79d-a9fa-4d4f-b4b1-f22292bd1294 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc5b5e637c9849fcb05a669133eef001)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:39.949827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666e79d-a9fa-4d4f-b4b1-f22292bd1294 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:40.008259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585271c-30c6-422f-a015-8d5a29deb580 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:40.009741 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b585271c-30c6-422f-a015-8d5a29deb580 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:40.022907 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585271c-30c6-422f-a015-8d5a29deb580 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:40.028116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece71e9d-34b7-4cfc-a83d-87e2e98eb4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:40.028975 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece71e9d-34b7-4cfc-a83d-87e2e98eb4aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:40.029255 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece71e9d-34b7-4cfc-a83d-87e2e98eb4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:40.029527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece71e9d-34b7-4cfc-a83d-87e2e98eb4aa admin admin] PATH_INFO: `/v3/users/6b9a202828a74bbdbb11973c5bf4bf96` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:40.030727 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece71e9d-34b7-4cfc-a83d-87e2e98eb4aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b9a202828a74bbdbb11973c5bf4bf96)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:40.032642 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece71e9d-34b7-4cfc-a83d-87e2e98eb4aa admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:40.209535 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-5666e79d-a9fa-4d4f-b4b1-f22292bd1294 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc5b5e637c9849fcb05a669133eef001'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:40.217043 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-5666e79d-a9fa-4d4f-b4b1-f22292bd1294 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc5b5e637c9849fcb05a669133eef001'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:40.217725 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-5666e79d-a9fa-4d4f-b4b1-f22292bd1294 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc5b5e637c9849fcb05a669133eef001'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:40.218347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-5666e79d-a9fa-4d4f-b4b1-f22292bd1294 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc5b5e637c9849fcb05a669133eef001'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:40.232395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-5666e79d-a9fa-4d4f-b4b1-f22292bd1294 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc5b5e637c9849fcb05a669133eef001'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:40.246962 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ece71e9d-34b7-4cfc-a83d-87e2e98eb4aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b9a202828a74bbdbb11973c5bf4bf96'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:40.254155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ece71e9d-34b7-4cfc-a83d-87e2e98eb4aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b9a202828a74bbdbb11973c5bf4bf96'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:40.254833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ece71e9d-34b7-4cfc-a83d-87e2e98eb4aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b9a202828a74bbdbb11973c5bf4bf96'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:40.255438 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ece71e9d-34b7-4cfc-a83d-87e2e98eb4aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b9a202828a74bbdbb11973c5bf4bf96'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:40.267056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ece71e9d-34b7-4cfc-a83d-87e2e98eb4aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b9a202828a74bbdbb11973c5bf4bf96'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:40.349268 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1092/2169] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:46:39 2022] DELETE /identity/v3/users/fc5b5e637c9849fcb05a669133eef001 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:40.354467 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4327027-b583-4925-b4dc-b3e85dab399d None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:40.354793 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4327027-b583-4925-b4dc-b3e85dab399d None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:40.367120 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4327027-b583-4925-b4dc-b3e85dab399d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:40.394667 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-b4327027-b583-4925-b4dc-b3e85dab399d None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:40.395899 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b4327027-b583-4925-b4dc-b3e85dab399d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:40.416057 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:40.454404 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1093/2170] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:40.464876 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4327027-b583-4925-b4dc-b3e85dab399d None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:40.466214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b4327027-b583-4925-b4dc-b3e85dab399d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:40.497693 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1078/2171] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:46:40 2022] DELETE /identity/v3/users/6b9a202828a74bbdbb11973c5bf4bf96 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:40.499494 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f980e-97c4-447b-8f6a-224b3bedfc48 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:40.500300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f980e-97c4-447b-8f6a-224b3bedfc48 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:40.500300 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f980e-97c4-447b-8f6a-224b3bedfc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:40.521094 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-7e0f980e-97c4-447b-8f6a-224b3bedfc48 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:40.522426 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7e0f980e-97c4-447b-8f6a-224b3bedfc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:40.559712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4327027-b583-4925-b4dc-b3e85dab399d None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:40.583388 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1079/2172] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:40.593765 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f980e-97c4-447b-8f6a-224b3bedfc48 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:40.607660 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:40.615020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b592d3-8ab3-4cac-9af7-49d4fa567a45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:40.615969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b592d3-8ab3-4cac-9af7-49d4fa567a45 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:40.615969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b592d3-8ab3-4cac-9af7-49d4fa567a45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:40.620238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b592d3-8ab3-4cac-9af7-49d4fa567a45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:40.635500 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b592d3-8ab3-4cac-9af7-49d4fa567a45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:40.637248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b592d3-8ab3-4cac-9af7-49d4fa567a45 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:40.648610 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1094/2173] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:46:40.654421 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f980e-97c4-447b-8f6a-224b3bedfc48 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:40.658246 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91df758-2b92-4f1d-85d9-443cee8701e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:40.658983 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91df758-2b92-4f1d-85d9-443cee8701e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:40.659272 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91df758-2b92-4f1d-85d9-443cee8701e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:40.659549 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91df758-2b92-4f1d-85d9-443cee8701e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:40.670115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91df758-2b92-4f1d-85d9-443cee8701e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:40.671710 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91df758-2b92-4f1d-85d9-443cee8701e6 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:40.684197 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1080/2174] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:46:41.410428 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b592d3-8ab3-4cac-9af7-49d4fa567a45 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:41.413520 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-22b592d3-8ab3-4cac-9af7-49d4fa567a45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:41.426055 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91df758-2b92-4f1d-85d9-443cee8701e6 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:41.427553 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f91df758-2b92-4f1d-85d9-443cee8701e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:41.438485 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b592d3-8ab3-4cac-9af7-49d4fa567a45 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:41.447974 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c527ac-9e97-469e-8bee-e7fda041592a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:41.448865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c527ac-9e97-469e-8bee-e7fda041592a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:41.449157 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c527ac-9e97-469e-8bee-e7fda041592a admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:41.449464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c527ac-9e97-469e-8bee-e7fda041592a admin admin] PATH_INFO: `/v3/projects/1508ad150f004c739d9f681c81648268` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:41.452280 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c527ac-9e97-469e-8bee-e7fda041592a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1508ad150f004c739d9f681c81648268)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:41.460049 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c527ac-9e97-469e-8bee-e7fda041592a admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:41.460547 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91df758-2b92-4f1d-85d9-443cee8701e6 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:41.467678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9cbd7-a6ca-4bdb-8f07-195263aedead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:41.467678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9cbd7-a6ca-4bdb-8f07-195263aedead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:41.467678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9cbd7-a6ca-4bdb-8f07-195263aedead admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:41.467678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9cbd7-a6ca-4bdb-8f07-195263aedead admin admin] PATH_INFO: `/v3/projects/c099614d2cf0413eae88fde46a1be0c8` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:41.468173 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a9cbd7-a6ca-4bdb-8f07-195263aedead admin admin] RBAC: Authorizing `identity:delete_project(project_id=c099614d2cf0413eae88fde46a1be0c8)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:41.475888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a9cbd7-a6ca-4bdb-8f07-195263aedead admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:41.685597 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-75c527ac-9e97-469e-8bee-e7fda041592a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1508ad150f004c739d9f681c81648268', 'request_id': 'req-75c527ac-9e97-469e-8bee-e7fda041592a'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:41.687524 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-27a9cbd7-a6ca-4bdb-8f07-195263aedead admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c099614d2cf0413eae88fde46a1be0c8', 'request_id': 'req-27a9cbd7-a6ca-4bdb-8f07-195263aedead'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:41.692975 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-75c527ac-9e97-469e-8bee-e7fda041592a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1508ad150f004c739d9f681c81648268', 'request_id': 'req-75c527ac-9e97-469e-8bee-e7fda041592a'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:41.696934 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-27a9cbd7-a6ca-4bdb-8f07-195263aedead admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c099614d2cf0413eae88fde46a1be0c8', 'request_id': 'req-27a9cbd7-a6ca-4bdb-8f07-195263aedead'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:41.828795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-75c527ac-9e97-469e-8bee-e7fda041592a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1508ad150f004c739d9f681c81648268', 'request_id': 'req-75c527ac-9e97-469e-8bee-e7fda041592a'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:41.830498 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:41.840805 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-75c527ac-9e97-469e-8bee-e7fda041592a admin admin] The token cache is being invalidate because project 1508ad150f004c739d9f681c81648268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:46:41.841090 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-75c527ac-9e97-469e-8bee-e7fda041592a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:41.841732 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-75c527ac-9e97-469e-8bee-e7fda041592a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:41.843409 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1095/2175] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:46:41 2022] DELETE /identity/v3/projects/1508ad150f004c739d9f681c81648268 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:41.848340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09f708-805f-46e0-bdea-f2f2721088d9 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:41.848660 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09f708-805f-46e0-bdea-f2f2721088d9 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:41.849146 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09f708-805f-46e0-bdea-f2f2721088d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:41.904081 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-2f09f708-805f-46e0-bdea-f2f2721088d9 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:41.905333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-2f09f708-805f-46e0-bdea-f2f2721088d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:41.918136 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-27a9cbd7-a6ca-4bdb-8f07-195263aedead admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c099614d2cf0413eae88fde46a1be0c8', 'request_id': 'req-27a9cbd7-a6ca-4bdb-8f07-195263aedead'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:41.947223 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-27a9cbd7-a6ca-4bdb-8f07-195263aedead admin admin] The token cache is being invalidate because project c099614d2cf0413eae88fde46a1be0c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:46:41.947572 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-27a9cbd7-a6ca-4bdb-8f07-195263aedead admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:41.948239 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-27a9cbd7-a6ca-4bdb-8f07-195263aedead admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:41.951212 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1081/2176] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:46:41 2022] DELETE /identity/v3/projects/c099614d2cf0413eae88fde46a1be0c8 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:41.962664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf51e71-711d-4716-9f7a-a23d2ad93a02 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:41.962938 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf51e71-711d-4716-9f7a-a23d2ad93a02 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:41.963224 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf51e71-711d-4716-9f7a-a23d2ad93a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:41.985562 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-9cf51e71-711d-4716-9f7a-a23d2ad93a02 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:41.987035 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9cf51e71-711d-4716-9f7a-a23d2ad93a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:42.003764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1096/2177] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:46:42.014609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf0e73-2ba3-4915-919c-d6b5f13bb43b None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:42.014609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf0e73-2ba3-4915-919c-d6b5f13bb43b None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:42.014609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf0e73-2ba3-4915-919c-d6b5f13bb43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:42.038189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-80cf0e73-2ba3-4915-919c-d6b5f13bb43b None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:42.038189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-80cf0e73-2ba3-4915-919c-d6b5f13bb43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:42.049094 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1097/2178] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:46:42.053218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf0e73-2ba3-4915-919c-d6b5f13bb43b None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:42.065229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf0e73-2ba3-4915-919c-d6b5f13bb43b None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:42.071843 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85943253-6149-42a9-b519-b231317cd375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:42.072526 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85943253-6149-42a9-b519-b231317cd375 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:42.072891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85943253-6149-42a9-b519-b231317cd375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:42.073115 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85943253-6149-42a9-b519-b231317cd375 admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:42.073864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85943253-6149-42a9-b519-b231317cd375 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:46:42.074183 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85943253-6149-42a9-b519-b231317cd375 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:42.075529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1082/2179] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:42.075876 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85943253-6149-42a9-b519-b231317cd375 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:42.090372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd25096-e988-4f50-b8e1-a38f6e7fec4e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:42.090372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd25096-e988-4f50-b8e1-a38f6e7fec4e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:42.092048 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd25096-e988-4f50-b8e1-a38f6e7fec4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:42.092142 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1098/2180] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:46:42.098404 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85943253-6149-42a9-b519-b231317cd375 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:42.110865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85943253-6149-42a9-b519-b231317cd375 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:42.114404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ddd25096-e988-4f50-b8e1-a38f6e7fec4e None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:42.116274 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ddd25096-e988-4f50-b8e1-a38f6e7fec4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:42.116770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d2852-6e1b-46ff-a356-e996bf869332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:42.117472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d2852-6e1b-46ff-a356-e996bf869332 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:42.118882 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d2852-6e1b-46ff-a356-e996bf869332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:42.118882 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d2852-6e1b-46ff-a356-e996bf869332 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:42.120325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d2852-6e1b-46ff-a356-e996bf869332 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:42.122296 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d2852-6e1b-46ff-a356-e996bf869332 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:42.126843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1083/2181] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:42.130876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd25096-e988-4f50-b8e1-a38f6e7fec4e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:42.142604 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd25096-e988-4f50-b8e1-a38f6e7fec4e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:42.148288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c463c28-1439-4a65-a9f1-89bd2266f9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:42.149033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c463c28-1439-4a65-a9f1-89bd2266f9fa admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:42.149537 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c463c28-1439-4a65-a9f1-89bd2266f9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:42.150142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c463c28-1439-4a65-a9f1-89bd2266f9fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:42.151040 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c463c28-1439-4a65-a9f1-89bd2266f9fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:46:42.157350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c463c28-1439-4a65-a9f1-89bd2266f9fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:42.157350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c463c28-1439-4a65-a9f1-89bd2266f9fa admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:42.172765 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1084/2182] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:46:42.174082 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c463c28-1439-4a65-a9f1-89bd2266f9fa admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:42.185559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c463c28-1439-4a65-a9f1-89bd2266f9fa admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:42.191572 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f747219-bde2-4653-b704-913cf2c28d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:42.192297 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f747219-bde2-4653-b704-913cf2c28d92 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:42.192650 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f747219-bde2-4653-b704-913cf2c28d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:42.192928 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f747219-bde2-4653-b704-913cf2c28d92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:42.193743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f747219-bde2-4653-b704-913cf2c28d92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:42.195723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f747219-bde2-4653-b704-913cf2c28d92 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:42.219871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1099/2183] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:46:42 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:46:42.220649 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:42.221254 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d2852-6e1b-46ff-a356-e996bf869332 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:42.290337 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d2852-6e1b-46ff-a356-e996bf869332 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:42.298713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c10b8b-75dc-446e-8002-77d0bad5a67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:42.298912 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1085/2184] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:46:42 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:46:42.299033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c10b8b-75dc-446e-8002-77d0bad5a67a admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:42.299033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c10b8b-75dc-446e-8002-77d0bad5a67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:42.299535 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c10b8b-75dc-446e-8002-77d0bad5a67a admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:42.300187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c10b8b-75dc-446e-8002-77d0bad5a67a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:42.300313 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f747219-bde2-4653-b704-913cf2c28d92 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:42.302317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c10b8b-75dc-446e-8002-77d0bad5a67a admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:42.383046 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f747219-bde2-4653-b704-913cf2c28d92 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:42.388379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828efeea-aeb4-430e-9b19-d7a7865e3be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:42.389222 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828efeea-aeb4-430e-9b19-d7a7865e3be1 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:42.389607 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828efeea-aeb4-430e-9b19-d7a7865e3be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:42.390166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828efeea-aeb4-430e-9b19-d7a7865e3be1 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:42.391678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828efeea-aeb4-430e-9b19-d7a7865e3be1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:42.396030 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828efeea-aeb4-430e-9b19-d7a7865e3be1 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:42.416338 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1100/2185] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:46:42 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:46:42.417868 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c10b8b-75dc-446e-8002-77d0bad5a67a admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:42.428441 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c10b8b-75dc-446e-8002-77d0bad5a67a admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:42.433443 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abce476-7642-4941-8424-47e74d81ffd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:42.438024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abce476-7642-4941-8424-47e74d81ffd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:42.438024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abce476-7642-4941-8424-47e74d81ffd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:42.438024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abce476-7642-4941-8424-47e74d81ffd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:42.438024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abce476-7642-4941-8424-47e74d81ffd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:42.438024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abce476-7642-4941-8424-47e74d81ffd2 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:42.447951 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1101/2186] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:46:42.454633 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abce476-7642-4941-8424-47e74d81ffd2 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:42.458873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:42.464675 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abce476-7642-4941-8424-47e74d81ffd2 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:42.468708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1a32d-d2c0-4bff-83b1-e064ef06c46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:42.469377 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1a32d-d2c0-4bff-83b1-e064ef06c46e admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:42.469788 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1a32d-d2c0-4bff-83b1-e064ef06c46e admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:42.471813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1a32d-d2c0-4bff-83b1-e064ef06c46e admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:42.471813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c1a32d-d2c0-4bff-83b1-e064ef06c46e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:42.472464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c1a32d-d2c0-4bff-83b1-e064ef06c46e admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:42.489488 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1102/2187] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:46:42.494183 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1a32d-d2c0-4bff-83b1-e064ef06c46e admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:42.500217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1086/2188] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:46:42 2022] POST /identity/v3/users => generated 289 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:46:42.504271 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828efeea-aeb4-430e-9b19-d7a7865e3be1 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:42.507409 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1a32d-d2c0-4bff-83b1-e064ef06c46e admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:42.517001 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828efeea-aeb4-430e-9b19-d7a7865e3be1 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:42.517464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4dfb4f-f91f-4a79-b85b-eae8d74a0b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:42.518442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4dfb4f-f91f-4a79-b85b-eae8d74a0b0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:42.518743 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4dfb4f-f91f-4a79-b85b-eae8d74a0b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:42.519018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4dfb4f-f91f-4a79-b85b-eae8d74a0b0d admin admin] PATH_INFO: `/v3/projects/eb2187f9489844ab8bbf1f69df83b2de/users/5e0cc563017646f6acea9127f7f525ee/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:42.522129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b30bfc-42c9-4e4c-8da3-1522c6957495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:42.522897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b30bfc-42c9-4e4c-8da3-1522c6957495 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:42.523378 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b30bfc-42c9-4e4c-8da3-1522c6957495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:42.524372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b30bfc-42c9-4e4c-8da3-1522c6957495 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:42.525493 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b30bfc-42c9-4e4c-8da3-1522c6957495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:42.530603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b30bfc-42c9-4e4c-8da3-1522c6957495 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:42.536485 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4dfb4f-f91f-4a79-b85b-eae8d74a0b0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb2187f9489844ab8bbf1f69df83b2de, user_id=5e0cc563017646f6acea9127f7f525ee, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:42.542698 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1087/2189] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:46:42.542853 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4dfb4f-f91f-4a79-b85b-eae8d74a0b0d admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:42.544638 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b30bfc-42c9-4e4c-8da3-1522c6957495 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:42.555050 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b30bfc-42c9-4e4c-8da3-1522c6957495 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:42.560482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d69647-ef95-4f83-9d17-0c921e96b3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:42.561205 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d69647-ef95-4f83-9d17-0c921e96b3df admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:42.561666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d69647-ef95-4f83-9d17-0c921e96b3df admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:42.562005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d69647-ef95-4f83-9d17-0c921e96b3df admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:42.562845 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d69647-ef95-4f83-9d17-0c921e96b3df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:42.564548 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d69647-ef95-4f83-9d17-0c921e96b3df admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:42.577394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1088/2190] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:46:42.587185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d69647-ef95-4f83-9d17-0c921e96b3df admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:42.592375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d69647-ef95-4f83-9d17-0c921e96b3df admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:42.598105 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b69fd-597d-4e91-ab36-a72351a98eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:42.599238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b69fd-597d-4e91-ab36-a72351a98eec admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:42.599621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b69fd-597d-4e91-ab36-a72351a98eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:42.599983 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b69fd-597d-4e91-ab36-a72351a98eec admin admin] PATH_INFO: `/v3/projects/2e810a9bc3b44c4aa8ca2ddd0fa68f7b/users/6cc5a4676f6a416cbdc8c67ebd80cdf3/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:42.617316 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b69fd-597d-4e91-ab36-a72351a98eec admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e810a9bc3b44c4aa8ca2ddd0fa68f7b, user_id=6cc5a4676f6a416cbdc8c67ebd80cdf3, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:42.620097 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b69fd-597d-4e91-ab36-a72351a98eec admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:42.636747 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1103/2191] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:46:42 2022] PUT /identity/v3/projects/eb2187f9489844ab8bbf1f69df83b2de/users/5e0cc563017646f6acea9127f7f525ee/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:46:42.641790 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935dc57d-2460-48b4-8bea-a9509eaacf66 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:42.642086 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935dc57d-2460-48b4-8bea-a9509eaacf66 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:42.642368 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935dc57d-2460-48b4-8bea-a9509eaacf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:42.661910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-935dc57d-2460-48b4-8bea-a9509eaacf66 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:42.663141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-935dc57d-2460-48b4-8bea-a9509eaacf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:42.727779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1104/2192] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:46:42.740503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935dc57d-2460-48b4-8bea-a9509eaacf66 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:42.742353 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-935dc57d-2460-48b4-8bea-a9509eaacf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:42.750779 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1089/2193] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:46:42 2022] PUT /identity/v3/projects/2e810a9bc3b44c4aa8ca2ddd0fa68f7b/users/6cc5a4676f6a416cbdc8c67ebd80cdf3/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:46:42.763376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311a12f-ecaa-449c-8a23-2da0eb165854 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:42.763671 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311a12f-ecaa-449c-8a23-2da0eb165854 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:42.764401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311a12f-ecaa-449c-8a23-2da0eb165854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:42.789564 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3311a12f-ecaa-449c-8a23-2da0eb165854 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:42.791280 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3311a12f-ecaa-449c-8a23-2da0eb165854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:42.807225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935dc57d-2460-48b4-8bea-a9509eaacf66 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:42.873084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788d2a1-1933-4eb7-a60b-39a44b19c6b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:42.873750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788d2a1-1933-4eb7-a60b-39a44b19c6b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:42.874038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788d2a1-1933-4eb7-a60b-39a44b19c6b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:42.874315 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788d2a1-1933-4eb7-a60b-39a44b19c6b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:42.884517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1090/2194] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:42.886962 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e788d2a1-1933-4eb7-a60b-39a44b19c6b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:42.888728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e788d2a1-1933-4eb7-a60b-39a44b19c6b8 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:42.891563 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311a12f-ecaa-449c-8a23-2da0eb165854 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:42.901376 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1105/2195] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:46:42.903605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311a12f-ecaa-449c-8a23-2da0eb165854 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:42.907567 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de4875-34b6-4ee6-a8ff-e1aa67157098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:42.908609 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de4875-34b6-4ee6-a8ff-e1aa67157098 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:42.908997 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de4875-34b6-4ee6-a8ff-e1aa67157098 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:42.909362 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de4875-34b6-4ee6-a8ff-e1aa67157098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:42.919722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de4875-34b6-4ee6-a8ff-e1aa67157098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:42.921222 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de4875-34b6-4ee6-a8ff-e1aa67157098 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:42.934179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1091/2196] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:44.142489 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788d2a1-1933-4eb7-a60b-39a44b19c6b8 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:44.143826 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e788d2a1-1933-4eb7-a60b-39a44b19c6b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:44.160869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788d2a1-1933-4eb7-a60b-39a44b19c6b8 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:44.168105 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893e632-e531-4912-8b1f-31247e1b6456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:44.169227 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893e632-e531-4912-8b1f-31247e1b6456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:44.169612 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893e632-e531-4912-8b1f-31247e1b6456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:44.170032 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893e632-e531-4912-8b1f-31247e1b6456 admin admin] PATH_INFO: `/v3/users/036fcf3b2b5d4223a44765dffc92a661` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:44.171897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893e632-e531-4912-8b1f-31247e1b6456 admin admin] RBAC: Authorizing `identity:delete_user(user_id=036fcf3b2b5d4223a44765dffc92a661)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:44.174373 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893e632-e531-4912-8b1f-31247e1b6456 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:44.223283 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:44.443961 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-2893e632-e531-4912-8b1f-31247e1b6456 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '036fcf3b2b5d4223a44765dffc92a661'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:44.454414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-2893e632-e531-4912-8b1f-31247e1b6456 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '036fcf3b2b5d4223a44765dffc92a661'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:44.458618 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-2893e632-e531-4912-8b1f-31247e1b6456 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '036fcf3b2b5d4223a44765dffc92a661'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:44.459910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-2893e632-e531-4912-8b1f-31247e1b6456 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '036fcf3b2b5d4223a44765dffc92a661'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:44.478787 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-2893e632-e531-4912-8b1f-31247e1b6456 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '036fcf3b2b5d4223a44765dffc92a661'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:44.583865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1106/2197] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:46:44 2022] DELETE /identity/v3/users/036fcf3b2b5d4223a44765dffc92a661 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:44.589659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0616675-753a-4908-a76b-50811584baaa None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:44.590095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0616675-753a-4908-a76b-50811584baaa None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:44.590207 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0616675-753a-4908-a76b-50811584baaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:44.617663 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a0616675-753a-4908-a76b-50811584baaa None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:44.619578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a0616675-753a-4908-a76b-50811584baaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:44.642626 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:44.706064 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1092/2198] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:44.718245 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893e632-e531-4912-8b1f-31247e1b6456 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:44.721111 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-2893e632-e531-4912-8b1f-31247e1b6456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:44.801011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893e632-e531-4912-8b1f-31247e1b6456 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:44.808199 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56078f87-7928-4a2a-8833-694b1a0e1858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:44.810366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56078f87-7928-4a2a-8833-694b1a0e1858 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:44.811207 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56078f87-7928-4a2a-8833-694b1a0e1858 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:44.811500 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56078f87-7928-4a2a-8833-694b1a0e1858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:44.827694 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56078f87-7928-4a2a-8833-694b1a0e1858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:44.829407 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56078f87-7928-4a2a-8833-694b1a0e1858 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:44.851235 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1107/2199] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:44.860568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a67ea-d4b8-485e-b366-a3ef4f22bf42 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:44.861205 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a67ea-d4b8-485e-b366-a3ef4f22bf42 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:44.861622 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a67ea-d4b8-485e-b366-a3ef4f22bf42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:44.902138 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-9d9a67ea-d4b8-485e-b366-a3ef4f22bf42 None None] MFA Rules not processed for user `5e0cc563017646f6acea9127f7f525ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:44.903394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9d9a67ea-d4b8-485e-b366-a3ef4f22bf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:44.985924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1093/2200] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:44.998016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56078f87-7928-4a2a-8833-694b1a0e1858 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:45.000216 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-56078f87-7928-4a2a-8833-694b1a0e1858 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:45.074951 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56078f87-7928-4a2a-8833-694b1a0e1858 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:45.081269 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee1133-be2a-48e3-97a4-9c20ec23571c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:45.082192 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee1133-be2a-48e3-97a4-9c20ec23571c service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:45.082911 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee1133-be2a-48e3-97a4-9c20ec23571c service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:45.082911 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee1133-be2a-48e3-97a4-9c20ec23571c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:45.095339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee1133-be2a-48e3-97a4-9c20ec23571c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:45.096940 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee1133-be2a-48e3-97a4-9c20ec23571c service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:45.107979 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1108/2201] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:45.436302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a67ea-d4b8-485e-b366-a3ef4f22bf42 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:45.437768 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9d9a67ea-d4b8-485e-b366-a3ef4f22bf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:45.453369 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a67ea-d4b8-485e-b366-a3ef4f22bf42 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:45.458342 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97aed7-1099-44b9-8260-d495f7a713f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:45.459515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97aed7-1099-44b9-8260-d495f7a713f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:45.459798 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97aed7-1099-44b9-8260-d495f7a713f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:45.460094 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97aed7-1099-44b9-8260-d495f7a713f5 admin admin] PATH_INFO: `/v3/projects/6a51336b0b3f47468fe32d36e6b282ee` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:45.462622 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97aed7-1099-44b9-8260-d495f7a713f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a51336b0b3f47468fe32d36e6b282ee)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:45.464609 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97aed7-1099-44b9-8260-d495f7a713f5 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:45.596936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7e97aed7-1099-44b9-8260-d495f7a713f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a51336b0b3f47468fe32d36e6b282ee', 'request_id': 'req-7e97aed7-1099-44b9-8260-d495f7a713f5'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:45.607212 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7e97aed7-1099-44b9-8260-d495f7a713f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a51336b0b3f47468fe32d36e6b282ee', 'request_id': 'req-7e97aed7-1099-44b9-8260-d495f7a713f5'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:45.714362 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7e97aed7-1099-44b9-8260-d495f7a713f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a51336b0b3f47468fe32d36e6b282ee', 'request_id': 'req-7e97aed7-1099-44b9-8260-d495f7a713f5'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:45.729578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7e97aed7-1099-44b9-8260-d495f7a713f5 admin admin] The token cache is being invalidate because project 6a51336b0b3f47468fe32d36e6b282ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:46:45.729939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7e97aed7-1099-44b9-8260-d495f7a713f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:45.730573 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7e97aed7-1099-44b9-8260-d495f7a713f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:45.732176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1094/2202] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:46:45 2022] DELETE /identity/v3/projects/6a51336b0b3f47468fe32d36e6b282ee => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:45.736485 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee1133-be2a-48e3-97a4-9c20ec23571c service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:45.750429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee1133-be2a-48e3-97a4-9c20ec23571c service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:45.816016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b333bc-b8b2-4e79-aa56-4b0fc199a661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:45.816942 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b333bc-b8b2-4e79-aa56-4b0fc199a661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:45.817254 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b333bc-b8b2-4e79-aa56-4b0fc199a661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:45.817531 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b333bc-b8b2-4e79-aa56-4b0fc199a661 admin admin] PATH_INFO: `/v3/users/0c27a222470f43a4b91d8e8f9d41db38` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:45.818759 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b333bc-b8b2-4e79-aa56-4b0fc199a661 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c27a222470f43a4b91d8e8f9d41db38)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:45.820762 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b333bc-b8b2-4e79-aa56-4b0fc199a661 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:46.093431 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:46.118401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-24b333bc-b8b2-4e79-aa56-4b0fc199a661 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c27a222470f43a4b91d8e8f9d41db38'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:46.139295 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-24b333bc-b8b2-4e79-aa56-4b0fc199a661 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c27a222470f43a4b91d8e8f9d41db38'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:46.140057 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-24b333bc-b8b2-4e79-aa56-4b0fc199a661 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c27a222470f43a4b91d8e8f9d41db38'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:46.140668 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-24b333bc-b8b2-4e79-aa56-4b0fc199a661 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c27a222470f43a4b91d8e8f9d41db38'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:46.165786 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-24b333bc-b8b2-4e79-aa56-4b0fc199a661 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c27a222470f43a4b91d8e8f9d41db38'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:46.267881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1109/2203] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:46:45 2022] DELETE /identity/v3/users/0c27a222470f43a4b91d8e8f9d41db38 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:46.827523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97aed7-1099-44b9-8260-d495f7a713f5 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:46.829319 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7e97aed7-1099-44b9-8260-d495f7a713f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:46.861065 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97aed7-1099-44b9-8260-d495f7a713f5 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:46.933645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:46.956663 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7bb05d-e87e-4729-90f6-3df0ebba993c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:46.957628 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7bb05d-e87e-4729-90f6-3df0ebba993c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:46.957913 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7bb05d-e87e-4729-90f6-3df0ebba993c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:46.958185 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7bb05d-e87e-4729-90f6-3df0ebba993c admin admin] PATH_INFO: `/v3/projects/fa1e73231aca44778c023d295a8f61c1` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:46.959723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7bb05d-e87e-4729-90f6-3df0ebba993c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa1e73231aca44778c023d295a8f61c1)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:46.962421 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7bb05d-e87e-4729-90f6-3df0ebba993c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:47.378162 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2f7bb05d-e87e-4729-90f6-3df0ebba993c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa1e73231aca44778c023d295a8f61c1', 'request_id': 'req-2f7bb05d-e87e-4729-90f6-3df0ebba993c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:47.385913 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2f7bb05d-e87e-4729-90f6-3df0ebba993c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa1e73231aca44778c023d295a8f61c1', 'request_id': 'req-2f7bb05d-e87e-4729-90f6-3df0ebba993c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:47.482645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2f7bb05d-e87e-4729-90f6-3df0ebba993c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa1e73231aca44778c023d295a8f61c1', 'request_id': 'req-2f7bb05d-e87e-4729-90f6-3df0ebba993c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:47.504777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2f7bb05d-e87e-4729-90f6-3df0ebba993c admin admin] The token cache is being invalidate because project fa1e73231aca44778c023d295a8f61c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:46:47.505024 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2f7bb05d-e87e-4729-90f6-3df0ebba993c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:47.511234 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2f7bb05d-e87e-4729-90f6-3df0ebba993c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:47.512198 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1095/2204] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:46:46 2022] DELETE /identity/v3/projects/fa1e73231aca44778c023d295a8f61c1 => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:47.518782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02304a6a-4a7b-4b99-9253-765fb59c6560 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:47.519860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02304a6a-4a7b-4b99-9253-765fb59c6560 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:47.520299 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02304a6a-4a7b-4b99-9253-765fb59c6560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:47.553739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-02304a6a-4a7b-4b99-9253-765fb59c6560 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:47.557333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-02304a6a-4a7b-4b99-9253-765fb59c6560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:47.694611 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1110/2205] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:47.701913 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bca48-a79a-4731-9098-9b1d5915b423 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:47.702412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bca48-a79a-4731-9098-9b1d5915b423 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:47.702860 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bca48-a79a-4731-9098-9b1d5915b423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:47.731200 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ac6bca48-a79a-4731-9098-9b1d5915b423 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:47.732475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ac6bca48-a79a-4731-9098-9b1d5915b423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:47.742925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1096/2206] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:47.746930 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02304a6a-4a7b-4b99-9253-765fb59c6560 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:47.758609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02304a6a-4a7b-4b99-9253-765fb59c6560 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:47.765220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d8acd-c353-414d-9a38-30ba1c15def2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:47.767095 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d8acd-c353-414d-9a38-30ba1c15def2 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:47.767593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d8acd-c353-414d-9a38-30ba1c15def2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:47.768040 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d8acd-c353-414d-9a38-30ba1c15def2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:47.769159 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d8acd-c353-414d-9a38-30ba1c15def2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:46:47.769647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d8acd-c353-414d-9a38-30ba1c15def2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:47.771912 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d8acd-c353-414d-9a38-30ba1c15def2 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:47.791492 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1111/2207] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:46:47.795650 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6bca48-a79a-4731-9098-9b1d5915b423 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:47.828713 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6bca48-a79a-4731-9098-9b1d5915b423 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:47.828713 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aecf34-fa75-4cd8-9cdc-72ae9cc18754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:47.828713 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aecf34-fa75-4cd8-9cdc-72ae9cc18754 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:47.828713 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aecf34-fa75-4cd8-9cdc-72ae9cc18754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:47.830505 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aecf34-fa75-4cd8-9cdc-72ae9cc18754 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:47.830505 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aecf34-fa75-4cd8-9cdc-72ae9cc18754 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:47.830505 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aecf34-fa75-4cd8-9cdc-72ae9cc18754 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:47.901172 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1097/2208] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:46:47 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:46:47.905767 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d8acd-c353-414d-9a38-30ba1c15def2 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:47.917291 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d8acd-c353-414d-9a38-30ba1c15def2 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:47.991606 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:48.003723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7556bd-5759-4c18-812a-98407c3a202a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:48.004584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7556bd-5759-4c18-812a-98407c3a202a admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:48.005011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7556bd-5759-4c18-812a-98407c3a202a admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:48.006248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7556bd-5759-4c18-812a-98407c3a202a admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:48.007876 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7556bd-5759-4c18-812a-98407c3a202a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:48.012263 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7556bd-5759-4c18-812a-98407c3a202a admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:48.157219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1112/2209] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:46:47 2022] POST /identity/v3/users => generated 293 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:46:48.158579 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aecf34-fa75-4cd8-9cdc-72ae9cc18754 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:48.171321 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aecf34-fa75-4cd8-9cdc-72ae9cc18754 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:48.179222 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba6666-407e-4363-a77a-a3ba094f990e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:48.181525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba6666-407e-4363-a77a-a3ba094f990e admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:48.181525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba6666-407e-4363-a77a-a3ba094f990e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:48.182130 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba6666-407e-4363-a77a-a3ba094f990e admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:48.183250 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba6666-407e-4363-a77a-a3ba094f990e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:48.185645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba6666-407e-4363-a77a-a3ba094f990e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:48.204530 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1098/2210] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:46:48.205512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7556bd-5759-4c18-812a-98407c3a202a admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:48.217873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7556bd-5759-4c18-812a-98407c3a202a admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:48.224938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399df11-64a4-435b-a4fd-bb0c5acabb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:48.224938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399df11-64a4-435b-a4fd-bb0c5acabb8f admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:48.225162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399df11-64a4-435b-a4fd-bb0c5acabb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:48.225425 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399df11-64a4-435b-a4fd-bb0c5acabb8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:48.226273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3399df11-64a4-435b-a4fd-bb0c5acabb8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:48.228225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3399df11-64a4-435b-a4fd-bb0c5acabb8f admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:48.245267 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1113/2211] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:46:48.247028 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba6666-407e-4363-a77a-a3ba094f990e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:48.262482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba6666-407e-4363-a77a-a3ba094f990e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:48.268463 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf721a-a625-46ec-ad84-cd43dd3b4e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:48.270149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf721a-a625-46ec-ad84-cd43dd3b4e48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:48.270628 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf721a-a625-46ec-ad84-cd43dd3b4e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:48.270903 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf721a-a625-46ec-ad84-cd43dd3b4e48 admin admin] PATH_INFO: `/v3/projects/053bded1036b4de2864f2ed404054185/users/60fe615480534283b7a0bb83e1df6742/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:48.280185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed64e6c-85e5-42b6-9403-a58f86cb7b60 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:48.283190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed64e6c-85e5-42b6-9403-a58f86cb7b60 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:48.285118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed64e6c-85e5-42b6-9403-a58f86cb7b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:48.291985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf721a-a625-46ec-ad84-cd43dd3b4e48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=053bded1036b4de2864f2ed404054185, user_id=60fe615480534283b7a0bb83e1df6742, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:48.302897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf721a-a625-46ec-ad84-cd43dd3b4e48 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:48.318827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-bed64e6c-85e5-42b6-9403-a58f86cb7b60 None None] MFA Rules not processed for user `5e0cc563017646f6acea9127f7f525ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:48.320334 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-bed64e6c-85e5-42b6-9403-a58f86cb7b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:48.360670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1099/2212] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:46:48 2022] PUT /identity/v3/projects/053bded1036b4de2864f2ed404054185/users/60fe615480534283b7a0bb83e1df6742/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:46:48.377250 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253369a3-4876-4245-973b-884d969defcc None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:48.377250 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253369a3-4876-4245-973b-884d969defcc None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:48.377548 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253369a3-4876-4245-973b-884d969defcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:48.406283 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1114/2213] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:48.414588 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-253369a3-4876-4245-973b-884d969defcc None None] MFA Rules not processed for user `60fe615480534283b7a0bb83e1df6742`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:48.416573 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-253369a3-4876-4245-973b-884d969defcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:48.521781 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1100/2214] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:48.533043 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed64e6c-85e5-42b6-9403-a58f86cb7b60 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:48.534500 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-bed64e6c-85e5-42b6-9403-a58f86cb7b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:48.648554 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed64e6c-85e5-42b6-9403-a58f86cb7b60 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:48.694996 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:48.733571 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7564b9f-7d7e-43fd-91ce-5ac75cc7bcec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:48.735399 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7564b9f-7d7e-43fd-91ce-5ac75cc7bcec service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:48.735797 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7564b9f-7d7e-43fd-91ce-5ac75cc7bcec service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:48.736080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7564b9f-7d7e-43fd-91ce-5ac75cc7bcec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:48.755662 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7564b9f-7d7e-43fd-91ce-5ac75cc7bcec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:48.757639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7564b9f-7d7e-43fd-91ce-5ac75cc7bcec service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:48.770431 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1115/2215] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:48.795826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253369a3-4876-4245-973b-884d969defcc None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:48.895882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253369a3-4876-4245-973b-884d969defcc None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:48.901571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66563dc-68d7-4850-8412-56ae9e7643e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:48.902572 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66563dc-68d7-4850-8412-56ae9e7643e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:48.902805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66563dc-68d7-4850-8412-56ae9e7643e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:48.905747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66563dc-68d7-4850-8412-56ae9e7643e4 admin admin] PATH_INFO: `/v3/users/60fe615480534283b7a0bb83e1df6742` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:48.905747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66563dc-68d7-4850-8412-56ae9e7643e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60fe615480534283b7a0bb83e1df6742)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:48.906223 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66563dc-68d7-4850-8412-56ae9e7643e4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:49.158020 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e66563dc-68d7-4850-8412-56ae9e7643e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60fe615480534283b7a0bb83e1df6742'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:49.173848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e66563dc-68d7-4850-8412-56ae9e7643e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60fe615480534283b7a0bb83e1df6742'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:49.174992 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e66563dc-68d7-4850-8412-56ae9e7643e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60fe615480534283b7a0bb83e1df6742'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:49.176299 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e66563dc-68d7-4850-8412-56ae9e7643e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60fe615480534283b7a0bb83e1df6742'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:49.198050 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e66563dc-68d7-4850-8412-56ae9e7643e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60fe615480534283b7a0bb83e1df6742'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:49.285430 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1101/2216] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:46:48 2022] DELETE /identity/v3/users/60fe615480534283b7a0bb83e1df6742 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:49.286903 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f26a0-35f4-433c-9f25-5baa53905c5c None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:49.287196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f26a0-35f4-433c-9f25-5baa53905c5c None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:49.287814 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f26a0-35f4-433c-9f25-5baa53905c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:49.314135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-ef7f26a0-35f4-433c-9f25-5baa53905c5c None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:49.316989 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-ef7f26a0-35f4-433c-9f25-5baa53905c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:49.458569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:49.471013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1116/2217] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:49.487042 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66563dc-68d7-4850-8412-56ae9e7643e4 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:49.488687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e66563dc-68d7-4850-8412-56ae9e7643e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:49.606433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66563dc-68d7-4850-8412-56ae9e7643e4 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:49.611596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78beb884-aeb9-48c0-b112-e9c39e846532 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:49.615083 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78beb884-aeb9-48c0-b112-e9c39e846532 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:49.615083 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78beb884-aeb9-48c0-b112-e9c39e846532 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:49.615083 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78beb884-aeb9-48c0-b112-e9c39e846532 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:49.631565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78beb884-aeb9-48c0-b112-e9c39e846532 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:49.634643 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78beb884-aeb9-48c0-b112-e9c39e846532 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:49.653561 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1102/2218] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:46:50.629144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f26a0-35f4-433c-9f25-5baa53905c5c None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:50.644137 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-ef7f26a0-35f4-433c-9f25-5baa53905c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:50.666963 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f26a0-35f4-433c-9f25-5baa53905c5c None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:50.675893 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75c036-25d2-40a4-a15c-445e1c648f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:50.676960 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75c036-25d2-40a4-a15c-445e1c648f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:50.677255 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75c036-25d2-40a4-a15c-445e1c648f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:50.677538 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75c036-25d2-40a4-a15c-445e1c648f46 admin admin] PATH_INFO: `/v3/projects/053bded1036b4de2864f2ed404054185` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:50.680569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75c036-25d2-40a4-a15c-445e1c648f46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=053bded1036b4de2864f2ed404054185)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:50.683087 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75c036-25d2-40a4-a15c-445e1c648f46 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:50.900814 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6e75c036-25d2-40a4-a15c-445e1c648f46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '053bded1036b4de2864f2ed404054185', 'request_id': 'req-6e75c036-25d2-40a4-a15c-445e1c648f46'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:50.915971 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6e75c036-25d2-40a4-a15c-445e1c648f46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '053bded1036b4de2864f2ed404054185', 'request_id': 'req-6e75c036-25d2-40a4-a15c-445e1c648f46'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:51.024299 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6e75c036-25d2-40a4-a15c-445e1c648f46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '053bded1036b4de2864f2ed404054185', 'request_id': 'req-6e75c036-25d2-40a4-a15c-445e1c648f46'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:51.052212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6e75c036-25d2-40a4-a15c-445e1c648f46 admin admin] The token cache is being invalidate because project 053bded1036b4de2864f2ed404054185 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:46:51.052514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6e75c036-25d2-40a4-a15c-445e1c648f46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:51.053346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6e75c036-25d2-40a4-a15c-445e1c648f46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:51.062710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1117/2219] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:46:50 2022] DELETE /identity/v3/projects/053bded1036b4de2864f2ed404054185 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:51.064364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4234c26-abec-4b75-a7dd-ef437be15b19 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:51.064805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4234c26-abec-4b75-a7dd-ef437be15b19 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:51.064920 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4234c26-abec-4b75-a7dd-ef437be15b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:51.096815 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d4234c26-abec-4b75-a7dd-ef437be15b19 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:51.098679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d4234c26-abec-4b75-a7dd-ef437be15b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:51.119546 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:51.224756 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1103/2220] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:51.226701 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b2548-1b36-465c-9e60-8251f610815f None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:51.226936 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b2548-1b36-465c-9e60-8251f610815f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:51.227262 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b2548-1b36-465c-9e60-8251f610815f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:51.254505 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-615b2548-1b36-465c-9e60-8251f610815f None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:51.256560 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-615b2548-1b36-465c-9e60-8251f610815f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:51.275101 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1118/2221] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:51.279310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4234c26-abec-4b75-a7dd-ef437be15b19 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:51.298716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4234c26-abec-4b75-a7dd-ef437be15b19 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:51.313636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3e303-f7e1-49f3-be77-3613736d3e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:51.314615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3e303-f7e1-49f3-be77-3613736d3e9f admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:51.315267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3e303-f7e1-49f3-be77-3613736d3e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:51.315547 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3e303-f7e1-49f3-be77-3613736d3e9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:51.316507 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3e303-f7e1-49f3-be77-3613736d3e9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:46:51.316984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3e303-f7e1-49f3-be77-3613736d3e9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:51.318721 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3e303-f7e1-49f3-be77-3613736d3e9f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:51.350614 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1104/2222] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:46:51.351798 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b2548-1b36-465c-9e60-8251f610815f None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:51.364630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b2548-1b36-465c-9e60-8251f610815f None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:51.376033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb392d24-f055-4ad2-af9a-17ded935b3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:51.376033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb392d24-f055-4ad2-af9a-17ded935b3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:51.387269 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb392d24-f055-4ad2-af9a-17ded935b3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:51.387530 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb392d24-f055-4ad2-af9a-17ded935b3a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:51.395976 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb392d24-f055-4ad2-af9a-17ded935b3a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:51.402505 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ec257-d228-4836-b715-a892c17db3cf None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:51.410434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb392d24-f055-4ad2-af9a-17ded935b3a8 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:51.410708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ec257-d228-4836-b715-a892c17db3cf None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:51.410708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ec257-d228-4836-b715-a892c17db3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:51.489244 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-0c4ec257-d228-4836-b715-a892c17db3cf None None] MFA Rules not processed for user `6cc5a4676f6a416cbdc8c67ebd80cdf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:51.493474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-0c4ec257-d228-4836-b715-a892c17db3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:51.552877 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1119/2223] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:46:51 2022] POST /identity/v3/projects => generated 371 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:46:51.557982 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb392d24-f055-4ad2-af9a-17ded935b3a8 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:51.575366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1105/2224] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:51.579777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ec257-d228-4836-b715-a892c17db3cf None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:51.581346 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-0c4ec257-d228-4836-b715-a892c17db3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:51.595888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ec257-d228-4836-b715-a892c17db3cf None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:51.660224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb392d24-f055-4ad2-af9a-17ded935b3a8 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:51.666056 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88061435-b3b0-4703-ba5a-90ea1379b804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:51.667368 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88061435-b3b0-4703-ba5a-90ea1379b804 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:51.667883 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88061435-b3b0-4703-ba5a-90ea1379b804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:51.668297 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88061435-b3b0-4703-ba5a-90ea1379b804 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:51.669118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88061435-b3b0-4703-ba5a-90ea1379b804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:51.671309 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88061435-b3b0-4703-ba5a-90ea1379b804 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:51.679314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cf83c-6114-4ae1-96ef-017ae2b897be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:51.682668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cf83c-6114-4ae1-96ef-017ae2b897be admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:51.682668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cf83c-6114-4ae1-96ef-017ae2b897be admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:51.682668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cf83c-6114-4ae1-96ef-017ae2b897be admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:51.682668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5cf83c-6114-4ae1-96ef-017ae2b897be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:51.684064 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5cf83c-6114-4ae1-96ef-017ae2b897be admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:51.827794 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1120/2225] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:46:51 2022] POST /identity/v3/users => generated 286 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:46:51.828411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88061435-b3b0-4703-ba5a-90ea1379b804 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:51.840264 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88061435-b3b0-4703-ba5a-90ea1379b804 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:51.847181 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef01d7-4d24-4283-a073-931eda9b7862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:51.848155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef01d7-4d24-4283-a073-931eda9b7862 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:51.848549 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef01d7-4d24-4283-a073-931eda9b7862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:51.848929 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef01d7-4d24-4283-a073-931eda9b7862 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:51.849801 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef01d7-4d24-4283-a073-931eda9b7862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:51.852233 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef01d7-4d24-4283-a073-931eda9b7862 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:51.864854 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1121/2226] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:46:51.869838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef01d7-4d24-4283-a073-931eda9b7862 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:51.879652 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1106/2227] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:46:51 2022] POST /identity/v3/projects => generated 379 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:46:51.881877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cf83c-6114-4ae1-96ef-017ae2b897be admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:51.895478 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef01d7-4d24-4283-a073-931eda9b7862 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:51.900812 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:51.962590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cf83c-6114-4ae1-96ef-017ae2b897be admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:51.967810 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473b23d-f9a6-4052-bff7-3e6f76e66f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:51.968483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473b23d-f9a6-4052-bff7-3e6f76e66f2d admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:51.969295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473b23d-f9a6-4052-bff7-3e6f76e66f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:51.969295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473b23d-f9a6-4052-bff7-3e6f76e66f2d admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:51.969737 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473b23d-f9a6-4052-bff7-3e6f76e66f2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:51.971716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473b23d-f9a6-4052-bff7-3e6f76e66f2d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:51.973452 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b7847-77c4-4a1e-bb1d-423c33433b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:51.976363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b7847-77c4-4a1e-bb1d-423c33433b10 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:51.976937 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b7847-77c4-4a1e-bb1d-423c33433b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:51.977289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b7847-77c4-4a1e-bb1d-423c33433b10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:51.978128 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b7847-77c4-4a1e-bb1d-423c33433b10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:51.981540 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b7847-77c4-4a1e-bb1d-423c33433b10 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:51.994295 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1122/2228] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:46:51.998802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b7847-77c4-4a1e-bb1d-423c33433b10 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:52.010155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b7847-77c4-4a1e-bb1d-423c33433b10 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:52.015343 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06090b4b-dbf6-4dae-a454-e3a174b0c7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:52.016507 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06090b4b-dbf6-4dae-a454-e3a174b0c7ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:52.017174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06090b4b-dbf6-4dae-a454-e3a174b0c7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:52.017407 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06090b4b-dbf6-4dae-a454-e3a174b0c7ad admin admin] PATH_INFO: `/v3/projects/a0c72b191b8542d581d74127863ed2b3/users/3c42d282d84a45f6a50c62d8ea906e45/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:52.040295 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06090b4b-dbf6-4dae-a454-e3a174b0c7ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0c72b191b8542d581d74127863ed2b3, user_id=3c42d282d84a45f6a50c62d8ea906e45, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:52.043486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06090b4b-dbf6-4dae-a454-e3a174b0c7ad admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:52.075520 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1107/2229] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:46:51 2022] POST /identity/v3/users => generated 296 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:46:52.079268 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473b23d-f9a6-4052-bff7-3e6f76e66f2d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:52.090253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473b23d-f9a6-4052-bff7-3e6f76e66f2d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:52.094958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7af3d-cb47-4049-8e93-3d7a4aa092a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:52.095623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7af3d-cb47-4049-8e93-3d7a4aa092a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:52.095910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7af3d-cb47-4049-8e93-3d7a4aa092a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:52.096208 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7af3d-cb47-4049-8e93-3d7a4aa092a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:52.097072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7af3d-cb47-4049-8e93-3d7a4aa092a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:52.098701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7af3d-cb47-4049-8e93-3d7a4aa092a7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:52.111894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1108/2230] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:46:52.116710 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7af3d-cb47-4049-8e93-3d7a4aa092a7 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:52.129253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7af3d-cb47-4049-8e93-3d7a4aa092a7 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:52.135312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c3047-683a-4164-9ba0-c8a0c1f50fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:52.136261 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c3047-683a-4164-9ba0-c8a0c1f50fc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:52.136546 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c3047-683a-4164-9ba0-c8a0c1f50fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:52.136819 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c3047-683a-4164-9ba0-c8a0c1f50fc5 admin admin] PATH_INFO: `/v3/projects/3827d72899c84273882f4b69098ae7b8/users/c8c82a19ef734e058ee04f21881b7f4b/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:52.155625 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c3047-683a-4164-9ba0-c8a0c1f50fc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3827d72899c84273882f4b69098ae7b8, user_id=c8c82a19ef734e058ee04f21881b7f4b, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:52.158302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c3047-683a-4164-9ba0-c8a0c1f50fc5 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:52.173334 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1123/2231] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:46:51 2022] PUT /identity/v3/projects/a0c72b191b8542d581d74127863ed2b3/users/3c42d282d84a45f6a50c62d8ea906e45/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:46:52.184013 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7226d7-d9bb-4c46-8661-b3f9ce2d1187 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:52.184642 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7226d7-d9bb-4c46-8661-b3f9ce2d1187 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:52.184999 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7226d7-d9bb-4c46-8661-b3f9ce2d1187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:52.213271 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-ef7226d7-d9bb-4c46-8661-b3f9ce2d1187 None None] MFA Rules not processed for user `3c42d282d84a45f6a50c62d8ea906e45`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:52.214474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-ef7226d7-d9bb-4c46-8661-b3f9ce2d1187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:52.252976 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1109/2232] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:46:52 2022] PUT /identity/v3/projects/3827d72899c84273882f4b69098ae7b8/users/c8c82a19ef734e058ee04f21881b7f4b/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:52.254439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c3047-683a-4164-9ba0-c8a0c1f50fc5 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:52.308769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1124/2233] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:46:52.321351 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7226d7-d9bb-4c46-8661-b3f9ce2d1187 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:52.323350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-ef7226d7-d9bb-4c46-8661-b3f9ce2d1187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:52.355787 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c3047-683a-4164-9ba0-c8a0c1f50fc5 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:52.361069 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:52.365014 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061c808-8a1c-4974-aaa0-fffd60b321b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:52.365852 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061c808-8a1c-4974-aaa0-fffd60b321b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:52.366233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061c808-8a1c-4974-aaa0-fffd60b321b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:52.366518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061c808-8a1c-4974-aaa0-fffd60b321b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:52.367319 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061c808-8a1c-4974-aaa0-fffd60b321b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:52.375402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061c808-8a1c-4974-aaa0-fffd60b321b9 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:52.380508 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1110/2234] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:46:52.386461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061c808-8a1c-4974-aaa0-fffd60b321b9 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:52.397797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061c808-8a1c-4974-aaa0-fffd60b321b9 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:52.404876 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7226d7-d9bb-4c46-8661-b3f9ce2d1187 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:52.405850 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc479bb-36de-4c71-a16b-0f9d86b5677e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:52.407301 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc479bb-36de-4c71-a16b-0f9d86b5677e admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:52.407687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc479bb-36de-4c71-a16b-0f9d86b5677e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:52.408071 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc479bb-36de-4c71-a16b-0f9d86b5677e admin admin] PATH_INFO: `/v3/projects/3827d72899c84273882f4b69098ae7b8/users/c8c82a19ef734e058ee04f21881b7f4b/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:52.410685 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc479bb-36de-4c71-a16b-0f9d86b5677e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3827d72899c84273882f4b69098ae7b8, user_id=c8c82a19ef734e058ee04f21881b7f4b, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:52.413886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc479bb-36de-4c71-a16b-0f9d86b5677e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:52.427217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1111/2235] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:46:52 2022] PUT /identity/v3/projects/3827d72899c84273882f4b69098ae7b8/users/c8c82a19ef734e058ee04f21881b7f4b/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:52.462801 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbaff3a-d930-43fd-9f6d-c064bf2a20cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:52.463458 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbaff3a-d930-43fd-9f6d-c064bf2a20cf service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:52.463740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbaff3a-d930-43fd-9f6d-c064bf2a20cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:52.464015 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbaff3a-d930-43fd-9f6d-c064bf2a20cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:52.549075 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbaff3a-d930-43fd-9f6d-c064bf2a20cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:52.551586 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbaff3a-d930-43fd-9f6d-c064bf2a20cf service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:52.563573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1125/2236] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:46:52.609025 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc479bb-36de-4c71-a16b-0f9d86b5677e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:52.755480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc479bb-36de-4c71-a16b-0f9d86b5677e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:52.762385 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b87527-9f60-4d30-989e-7a8aa2bfca1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:52.763320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b87527-9f60-4d30-989e-7a8aa2bfca1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:52.763839 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b87527-9f60-4d30-989e-7a8aa2bfca1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:52.764622 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b87527-9f60-4d30-989e-7a8aa2bfca1a admin admin] PATH_INFO: `/v3/users/3c42d282d84a45f6a50c62d8ea906e45` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:52.766116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b87527-9f60-4d30-989e-7a8aa2bfca1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c42d282d84a45f6a50c62d8ea906e45)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:52.768296 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b87527-9f60-4d30-989e-7a8aa2bfca1a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:53.017212 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-03b87527-9f60-4d30-989e-7a8aa2bfca1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c42d282d84a45f6a50c62d8ea906e45'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:53.032676 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-03b87527-9f60-4d30-989e-7a8aa2bfca1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c42d282d84a45f6a50c62d8ea906e45'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:53.033382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-03b87527-9f60-4d30-989e-7a8aa2bfca1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c42d282d84a45f6a50c62d8ea906e45'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:53.034003 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-03b87527-9f60-4d30-989e-7a8aa2bfca1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c42d282d84a45f6a50c62d8ea906e45'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:53.063519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:53.067086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-03b87527-9f60-4d30-989e-7a8aa2bfca1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c42d282d84a45f6a50c62d8ea906e45'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:53.150443 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1112/2237] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:46:52 2022] DELETE /identity/v3/users/3c42d282d84a45f6a50c62d8ea906e45 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:53.155076 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36555381-b897-4dcb-ba9d-9f8d3e6f7f76 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:53.155370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36555381-b897-4dcb-ba9d-9f8d3e6f7f76 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:53.155649 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36555381-b897-4dcb-ba9d-9f8d3e6f7f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:53.167085 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:53.177701 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-36555381-b897-4dcb-ba9d-9f8d3e6f7f76 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:53.179688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-36555381-b897-4dcb-ba9d-9f8d3e6f7f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:53.180286 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23cbaa-aea2-4983-aab0-a6fe5a6c49f1 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:53.180286 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23cbaa-aea2-4983-aab0-a6fe5a6c49f1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:53.181053 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23cbaa-aea2-4983-aab0-a6fe5a6c49f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:53.203098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-bd23cbaa-aea2-4983-aab0-a6fe5a6c49f1 None None] MFA Rules not processed for user `bcd5fc6167654972868941ed28c327b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:53.204319 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bd23cbaa-aea2-4983-aab0-a6fe5a6c49f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:53.280536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1126/2238] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:46:53.290091 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36555381-b897-4dcb-ba9d-9f8d3e6f7f76 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:53.292021 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-36555381-b897-4dcb-ba9d-9f8d3e6f7f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:53.324479 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1113/2239] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:46:53.447341 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36555381-b897-4dcb-ba9d-9f8d3e6f7f76 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:53.458240 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d03d6d4-976a-4552-92ce-53710cd4c0be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:53.460329 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03d6d4-976a-4552-92ce-53710cd4c0be service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:53.460329 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03d6d4-976a-4552-92ce-53710cd4c0be service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:53.460530 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03d6d4-976a-4552-92ce-53710cd4c0be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:53.476191 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03d6d4-976a-4552-92ce-53710cd4c0be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:53.483167 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03d6d4-976a-4552-92ce-53710cd4c0be service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:53.497060 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1127/2240] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:54.873175 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23cbaa-aea2-4983-aab0-a6fe5a6c49f1 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:54.875445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bd23cbaa-aea2-4983-aab0-a6fe5a6c49f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:54.902693 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23cbaa-aea2-4983-aab0-a6fe5a6c49f1 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:54.924394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a10989-544e-49b7-9130-ca5de8e27800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:54.925453 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a10989-544e-49b7-9130-ca5de8e27800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:54.925844 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a10989-544e-49b7-9130-ca5de8e27800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:54.926769 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a10989-544e-49b7-9130-ca5de8e27800 admin admin] PATH_INFO: `/v3/projects/a0c72b191b8542d581d74127863ed2b3` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:54.929607 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a10989-544e-49b7-9130-ca5de8e27800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0c72b191b8542d581d74127863ed2b3)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:54.933963 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a10989-544e-49b7-9130-ca5de8e27800 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:55.106375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d5a10989-544e-49b7-9130-ca5de8e27800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0c72b191b8542d581d74127863ed2b3', 'request_id': 'req-d5a10989-544e-49b7-9130-ca5de8e27800'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:55.116942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d5a10989-544e-49b7-9130-ca5de8e27800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0c72b191b8542d581d74127863ed2b3', 'request_id': 'req-d5a10989-544e-49b7-9130-ca5de8e27800'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:55.177148 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d5a10989-544e-49b7-9130-ca5de8e27800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0c72b191b8542d581d74127863ed2b3', 'request_id': 'req-d5a10989-544e-49b7-9130-ca5de8e27800'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:55.197869 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d5a10989-544e-49b7-9130-ca5de8e27800 admin admin] The token cache is being invalidate because project a0c72b191b8542d581d74127863ed2b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:46:55.199446 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d5a10989-544e-49b7-9130-ca5de8e27800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:55.202757 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d5a10989-544e-49b7-9130-ca5de8e27800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:46:55.210343 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1114/2241] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:46:54 2022] DELETE /identity/v3/projects/a0c72b191b8542d581d74127863ed2b3 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:46:59.855445 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b872a-5145-48c7-a213-db7566bc3ec8 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:59.855908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b872a-5145-48c7-a213-db7566bc3ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:59.855987 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b872a-5145-48c7-a213-db7566bc3ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:59.888093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-844b872a-5145-48c7-a213-db7566bc3ec8 None None] MFA Rules not processed for user `c8c82a19ef734e058ee04f21881b7f4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:59.889330 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-844b872a-5145-48c7-a213-db7566bc3ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:59.900071 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:59.963975 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1128/2242] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:46:59 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:59.977647 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a10989-544e-49b7-9130-ca5de8e27800 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:59.979438 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d5a10989-544e-49b7-9130-ca5de8e27800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:59.984318 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:47:00.103015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a10989-544e-49b7-9130-ca5de8e27800 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:00.108808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f5db8-4132-410e-8d4e-b643a52ee3a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:00.109408 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f5db8-4132-410e-8d4e-b643a52ee3a9 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:00.109691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f5db8-4132-410e-8d4e-b643a52ee3a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:00.110153 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f5db8-4132-410e-8d4e-b643a52ee3a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:00.112055 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-506f5db8-4132-410e-8d4e-b643a52ee3a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:00.176002 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f5db8-4132-410e-8d4e-b643a52ee3a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:00.177593 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f5db8-4132-410e-8d4e-b643a52ee3a9 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:00.189157 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1115/2243] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:47:10.268188 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=24 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:47:12.391523 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b872a-5145-48c7-a213-db7566bc3ec8 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:12.393029 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-844b872a-5145-48c7-a213-db7566bc3ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:12.452151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b872a-5145-48c7-a213-db7566bc3ec8 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:12.455714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344196b-d6aa-4d34-a390-0f3ec58549aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:12.456358 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344196b-d6aa-4d34-a390-0f3ec58549aa service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:12.456639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344196b-d6aa-4d34-a390-0f3ec58549aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:12.456914 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344196b-d6aa-4d34-a390-0f3ec58549aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:12.458696 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-4344196b-d6aa-4d34-a390-0f3ec58549aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:12.530524 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4344196b-d6aa-4d34-a390-0f3ec58549aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:12.531994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4344196b-d6aa-4d34-a390-0f3ec58549aa service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:12.542268 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1129/2244] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:12 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:47:12.628023 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=66 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:47:22.537115 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=50 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:47:35.427683 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2204e-0c3c-474b-b11b-4e325826b4b3 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:35.428188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2204e-0c3c-474b-b11b-4e325826b4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:35.428306 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2204e-0c3c-474b-b11b-4e325826b4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:35.445500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-8fb2204e-0c3c-474b-b11b-4e325826b4b3 None None] MFA Rules not processed for user `6cc5a4676f6a416cbdc8c67ebd80cdf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:35.446815 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-8fb2204e-0c3c-474b-b11b-4e325826b4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:35.454134 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1116/2245] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:47:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:37.936586 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344196b-d6aa-4d34-a390-0f3ec58549aa service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:37.938002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-4344196b-d6aa-4d34-a390-0f3ec58549aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:38.030333 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344196b-d6aa-4d34-a390-0f3ec58549aa service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:38.035414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a039a4-0239-4f10-b9c0-f19cef3c781c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac04b32777ca4f929e9e59deee6e0260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:38.036056 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a039a4-0239-4f10-b9c0-f19cef3c781c service placement] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:38.036341 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a039a4-0239-4f10-b9c0-f19cef3c781c service placement] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:38.036620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a039a4-0239-4f10-b9c0-f19cef3c781c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:38.038403 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b5a039a4-0239-4f10-b9c0-f19cef3c781c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:38.049729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a039a4-0239-4f10-b9c0-f19cef3c781c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:38.051468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a039a4-0239-4f10-b9c0-f19cef3c781c service placement] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:38.061563 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1130/2246] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:47:45.440746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:47:48.057311 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=41 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:47:54.059289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea52c96-5b93-4e0b-aaca-8c8eb205df7f None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:54.059874 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea52c96-5b93-4e0b-aaca-8c8eb205df7f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:54.059874 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea52c96-5b93-4e0b-aaca-8c8eb205df7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:54.080064 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-9ea52c96-5b93-4e0b-aaca-8c8eb205df7f None None] MFA Rules not processed for user `b7fd8323d29d445f903e2103453e8ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:54.081310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9ea52c96-5b93-4e0b-aaca-8c8eb205df7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:54.143925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1117/2247] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:47:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:57.109564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a039a4-0239-4f10-b9c0-f19cef3c781c service placement] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:57.111187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b5a039a4-0239-4f10-b9c0-f19cef3c781c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:57.122592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a039a4-0239-4f10-b9c0-f19cef3c781c service placement] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:57.127183 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230dc04-f5d7-44a2-a22e-6af124944912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:57.127901 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230dc04-f5d7-44a2-a22e-6af124944912 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:57.128248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230dc04-f5d7-44a2-a22e-6af124944912 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:57.128591 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230dc04-f5d7-44a2-a22e-6af124944912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:57.137931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3230dc04-f5d7-44a2-a22e-6af124944912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:57.140131 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3230dc04-f5d7-44a2-a22e-6af124944912 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:57.150206 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1131/2248] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:04.135875 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=25 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:48:07.146010 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=6 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:48:46.579733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59e5b7-0af3-4d5f-9daf-b89e7dceae7e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:46.580356 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59e5b7-0af3-4d5f-9daf-b89e7dceae7e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:46.580464 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59e5b7-0af3-4d5f-9daf-b89e7dceae7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:46.601425 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-2d59e5b7-0af3-4d5f-9daf-b89e7dceae7e None None] MFA Rules not processed for user `5e0cc563017646f6acea9127f7f525ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:46.602835 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2d59e5b7-0af3-4d5f-9daf-b89e7dceae7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:46.663620 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1118/2249] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:48:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:56.656335 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=25 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:49:02.624540 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230dc04-f5d7-44a2-a22e-6af124944912 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:49:02.632982 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230dc04-f5d7-44a2-a22e-6af124944912 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:49:02.636539 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7099d0-54f1-4e14-ac5b-6cca3779b529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:49:02.637164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7099d0-54f1-4e14-ac5b-6cca3779b529 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:02.637445 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7099d0-54f1-4e14-ac5b-6cca3779b529 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:02.637718 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7099d0-54f1-4e14-ac5b-6cca3779b529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:02.639494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-dc7099d0-54f1-4e14-ac5b-6cca3779b529 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:02.652943 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7099d0-54f1-4e14-ac5b-6cca3779b529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:49:02.654479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7099d0-54f1-4e14-ac5b-6cca3779b529 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:49:02.665870 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1132/2250] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:49:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:49:05.598437 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59e5b7-0af3-4d5f-9daf-b89e7dceae7e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:49:05.608068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59e5b7-0af3-4d5f-9daf-b89e7dceae7e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:49:05.612383 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf8bbc4-9d74-44e4-901d-20402a26292c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:49:05.613012 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8bbc4-9d74-44e4-901d-20402a26292c service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:05.613295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8bbc4-9d74-44e4-901d-20402a26292c service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:05.613568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8bbc4-9d74-44e4-901d-20402a26292c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:05.615337 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2bf8bbc4-9d74-44e4-901d-20402a26292c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:05.628406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf8bbc4-9d74-44e4-901d-20402a26292c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:49:05.629928 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf8bbc4-9d74-44e4-901d-20402a26292c service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:49:05.641199 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1119/2251] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:49:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:49:12.661009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=6 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:49:15.636204 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=6 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:49:29.170067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f8981-b2f7-4d1f-a922-96e3ea9401f8 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:29.170596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f8981-b2f7-4d1f-a922-96e3ea9401f8 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:29.170686 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f8981-b2f7-4d1f-a922-96e3ea9401f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:29.233265 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-263f8981-b2f7-4d1f-a922-96e3ea9401f8 None None] MFA Rules not processed for user `030614ef56f143f8be6fcc0445ec9568`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:49:29.234469 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-263f8981-b2f7-4d1f-a922-96e3ea9401f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:29.299130 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1133/2252] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 23 11:49:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:49:29.350242 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf8bbc4-9d74-44e4-901d-20402a26292c service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:49:29.351671 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2bf8bbc4-9d74-44e4-901d-20402a26292c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:29.448634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf8bbc4-9d74-44e4-901d-20402a26292c service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:49:29.453866 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f11adc-16ba-4d11-bfe2-272e34ba5caa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450b1fad96604fb09d0705895c6d3da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:49:29.454700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f11adc-16ba-4d11-bfe2-272e34ba5caa service swift] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:29.455017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f11adc-16ba-4d11-bfe2-272e34ba5caa service swift] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:29.460569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f11adc-16ba-4d11-bfe2-272e34ba5caa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:29.470386 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f11adc-16ba-4d11-bfe2-272e34ba5caa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:49:29.472025 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f11adc-16ba-4d11-bfe2-272e34ba5caa service swift] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:49:29.480743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1120/2253] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 23 11:49:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:49:39.295849 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=44 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:49:39.482129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=41 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:49:42.435938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f8981-b2f7-4d1f-a922-96e3ea9401f8 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:49:42.446379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f8981-b2f7-4d1f-a922-96e3ea9401f8 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:49:42.450004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276546d-ecc6-4433-a076-58466c3ba6ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:49:42.450680 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276546d-ecc6-4433-a076-58466c3ba6ef service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:42.450963 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276546d-ecc6-4433-a076-58466c3ba6ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:42.451239 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276546d-ecc6-4433-a076-58466c3ba6ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:42.453059 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-0276546d-ecc6-4433-a076-58466c3ba6ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:42.462688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276546d-ecc6-4433-a076-58466c3ba6ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:49:42.464209 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276546d-ecc6-4433-a076-58466c3ba6ef service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:49:42.472408 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1134/2254] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:49:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:49:48.815308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f11adc-16ba-4d11-bfe2-272e34ba5caa service swift] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:49:48.816853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a9f11adc-16ba-4d11-bfe2-272e34ba5caa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:48.832596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f11adc-16ba-4d11-bfe2-272e34ba5caa service swift] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:49:48.838285 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c753ad1-8ffa-4dcd-87ab-20f610dfcfca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:49:48.838983 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c753ad1-8ffa-4dcd-87ab-20f610dfcfca service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:48.839265 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c753ad1-8ffa-4dcd-87ab-20f610dfcfca service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:48.839545 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c753ad1-8ffa-4dcd-87ab-20f610dfcfca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:48.841977 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3c753ad1-8ffa-4dcd-87ab-20f610dfcfca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:48.901615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c753ad1-8ffa-4dcd-87ab-20f610dfcfca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:49:48.903241 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c753ad1-8ffa-4dcd-87ab-20f610dfcfca service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:49:48.914659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1121/2255] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:49:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:49:51.175875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276546d-ecc6-4433-a076-58466c3ba6ef service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:49:51.184447 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276546d-ecc6-4433-a076-58466c3ba6ef service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:49:51.187942 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf674ac4-05fc-48ff-a32e-a57faf68cb3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:49:51.188571 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf674ac4-05fc-48ff-a32e-a57faf68cb3c service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:51.188850 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf674ac4-05fc-48ff-a32e-a57faf68cb3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:51.189162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf674ac4-05fc-48ff-a32e-a57faf68cb3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:51.190859 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-bf674ac4-05fc-48ff-a32e-a57faf68cb3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:51.199986 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf674ac4-05fc-48ff-a32e-a57faf68cb3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:49:51.201427 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf674ac4-05fc-48ff-a32e-a57faf68cb3c service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:49:51.210099 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1135/2256] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:49:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:49:51.467901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153291e-7904-4250-804f-37a5fc6fc361 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:51.468395 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153291e-7904-4250-804f-37a5fc6fc361 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:51.468522 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153291e-7904-4250-804f-37a5fc6fc361 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:51.469857 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1122/2257] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 23 11:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:49:51.476076 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2b6b8-9705-4a21-97d5-5b9d5833b787 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:51.476446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2b6b8-9705-4a21-97d5-5b9d5833b787 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:51.476796 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2b6b8-9705-4a21-97d5-5b9d5833b787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:51.532732 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-03a2b6b8-9705-4a21-97d5-5b9d5833b787 None None] MFA Rules not processed for user `161c5ed1e47e450290a1f428ee100b68`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:49:51.534080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-03a2b6b8-9705-4a21-97d5-5b9d5833b787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:51.544160 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1136/2258] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 23 11:49:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:49:51.555529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153291e-7904-4250-804f-37a5fc6fc361 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:49:51.557218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9153291e-7904-4250-804f-37a5fc6fc361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:51.637573 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153291e-7904-4250-804f-37a5fc6fc361 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:49:51.641224 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04758f-be80-466b-948d-5913564ff674 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6105838cbf1409587ee7d9e3be554f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:49:51.641887 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04758f-be80-466b-948d-5913564ff674 service glance] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:51.642155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04758f-be80-466b-948d-5913564ff674 service glance] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:51.642434 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04758f-be80-466b-948d-5913564ff674 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:51.653622 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04758f-be80-466b-948d-5913564ff674 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:49:51.655143 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04758f-be80-466b-948d-5913564ff674 service glance] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:49:51.665984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1123/2259] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:49:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:49:51.850701 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf82b6-fa15-45fc-bf58-594887a67a0a None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:51.851128 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf82b6-fa15-45fc-bf58-594887a67a0a None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:51.851428 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf82b6-fa15-45fc-bf58-594887a67a0a None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:51.852952 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1137/2260] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 23 11:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:49:51.858333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32606172-726c-49f5-a10b-362d832b50bd None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:51.858833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32606172-726c-49f5-a10b-362d832b50bd None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:51.859172 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32606172-726c-49f5-a10b-362d832b50bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:51.879604 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-32606172-726c-49f5-a10b-362d832b50bd None None] MFA Rules not processed for user `810fa3f54cd04f989252765f7e76e602`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:49:51.880910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-32606172-726c-49f5-a10b-362d832b50bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:51.890494 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1124/2261] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 23 11:49:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:49:51.903403 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf82b6-fa15-45fc-bf58-594887a67a0a None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:49:51.914515 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf82b6-fa15-45fc-bf58-594887a67a0a None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:49:51.918630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96559659-9325-4e91-9d1e-7bd81c24fdfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:49:51.919554 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96559659-9325-4e91-9d1e-7bd81c24fdfe service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:51.919836 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96559659-9325-4e91-9d1e-7bd81c24fdfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:51.920582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96559659-9325-4e91-9d1e-7bd81c24fdfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:51.929666 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96559659-9325-4e91-9d1e-7bd81c24fdfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:49:51.931151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96559659-9325-4e91-9d1e-7bd81c24fdfe service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:49:51.946835 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1138/2262] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:49:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:49:52.030459 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb9cf3-8fd1-4487-9296-4a21e86dda9a None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:52.030789 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb9cf3-8fd1-4487-9296-4a21e86dda9a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:52.031068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb9cf3-8fd1-4487-9296-4a21e86dda9a None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:52.032512 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1125/2263] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 23 11:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:49:52.037665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b6715-2a4d-4a51-a43e-ac88968a4ea5 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:52.037953 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b6715-2a4d-4a51-a43e-ac88968a4ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:52.038233 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b6715-2a4d-4a51-a43e-ac88968a4ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:52.057045 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-662b6715-2a4d-4a51-a43e-ac88968a4ea5 None None] MFA Rules not processed for user `161c5ed1e47e450290a1f428ee100b68`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:49:52.058175 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-662b6715-2a4d-4a51-a43e-ac88968a4ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:52.067743 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1139/2264] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 23 11:49:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:49:52.080395 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb9cf3-8fd1-4487-9296-4a21e86dda9a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:49:52.091545 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb9cf3-8fd1-4487-9296-4a21e86dda9a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:49:52.095481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae415d2-b53c-4364-95e5-df865133d91a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:49:52.096257 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae415d2-b53c-4364-95e5-df865133d91a service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:52.096536 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae415d2-b53c-4364-95e5-df865133d91a service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:52.096809 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae415d2-b53c-4364-95e5-df865133d91a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:52.107470 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae415d2-b53c-4364-95e5-df865133d91a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:49:52.108883 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae415d2-b53c-4364-95e5-df865133d91a service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:49:52.120183 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1126/2265] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:49:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:49:52.189366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b6715-2a4d-4a51-a43e-ac88968a4ea5 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:49:52.200487 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b6715-2a4d-4a51-a43e-ac88968a4ea5 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:49:52.205537 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7140fc-41a5-4583-93b8-2e4a622795cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac04b32777ca4f929e9e59deee6e0260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:49:52.206171 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7140fc-41a5-4583-93b8-2e4a622795cc service placement] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:52.206459 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7140fc-41a5-4583-93b8-2e4a622795cc service placement] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:52.206768 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7140fc-41a5-4583-93b8-2e4a622795cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:52.208549 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-1b7140fc-41a5-4583-93b8-2e4a622795cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:52.220473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7140fc-41a5-4583-93b8-2e4a622795cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:49:52.222068 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7140fc-41a5-4583-93b8-2e4a622795cc service placement] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:49:52.232724 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1140/2266] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:49:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:50:02.115364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=72 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:50:02.228520 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=36 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:50:04.422698 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bb34c-f6bf-4f06-9160-56150bd47412 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:04.423170 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bb34c-f6bf-4f06-9160-56150bd47412 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:04.423298 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bb34c-f6bf-4f06-9160-56150bd47412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:04.441108 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-162bb34c-f6bf-4f06-9160-56150bd47412 None None] MFA Rules not processed for user `030614ef56f143f8be6fcc0445ec9568`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:50:04.442286 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-162bb34c-f6bf-4f06-9160-56150bd47412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:04.450041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1127/2267] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 23 11:50:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:50:04.462421 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7140fc-41a5-4583-93b8-2e4a622795cc service placement] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:04.475297 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7140fc-41a5-4583-93b8-2e4a622795cc service placement] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:04.481370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084e3af-34a6-4b8d-8106-8248a1eb6ba1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450b1fad96604fb09d0705895c6d3da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:04.482545 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084e3af-34a6-4b8d-8106-8248a1eb6ba1 service swift] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:04.489208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084e3af-34a6-4b8d-8106-8248a1eb6ba1 service swift] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:04.489208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084e3af-34a6-4b8d-8106-8248a1eb6ba1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:04.494119 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084e3af-34a6-4b8d-8106-8248a1eb6ba1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:04.495834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084e3af-34a6-4b8d-8106-8248a1eb6ba1 service swift] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:04.504701 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1141/2268] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 23 11:50:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:50:12.513514 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162bb34c-f6bf-4f06-9160-56150bd47412 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:12.536698 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162bb34c-f6bf-4f06-9160-56150bd47412 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:12.546094 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172297e-120e-4cfa-a7ee-408a9876ac4b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac04b32777ca4f929e9e59deee6e0260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:12.547453 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172297e-120e-4cfa-a7ee-408a9876ac4b service placement] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:12.548238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172297e-120e-4cfa-a7ee-408a9876ac4b service placement] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:12.548890 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172297e-120e-4cfa-a7ee-408a9876ac4b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:12.552293 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4172297e-120e-4cfa-a7ee-408a9876ac4b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:12.574739 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4172297e-120e-4cfa-a7ee-408a9876ac4b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:12.578193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4172297e-120e-4cfa-a7ee-408a9876ac4b service placement] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:12.596391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1128/2269] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:50:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:50:12.691911 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084e3af-34a6-4b8d-8106-8248a1eb6ba1 service swift] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:12.692689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e084e3af-34a6-4b8d-8106-8248a1eb6ba1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:12.778589 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084e3af-34a6-4b8d-8106-8248a1eb6ba1 service swift] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:12.783089 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e3295-a5af-4828-a1c5-e9d1ab92ab55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:12.783994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e3295-a5af-4828-a1c5-e9d1ab92ab55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:12.784277 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e3295-a5af-4828-a1c5-e9d1ab92ab55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:12.784717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e3295-a5af-4828-a1c5-e9d1ab92ab55 admin admin] PATH_INFO: `/v3/users/5e0cc563017646f6acea9127f7f525ee` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:12.785930 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e3295-a5af-4828-a1c5-e9d1ab92ab55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e0cc563017646f6acea9127f7f525ee)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:12.788020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e3295-a5af-4828-a1c5-e9d1ab92ab55 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:13.012317 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-459e3295-a5af-4828-a1c5-e9d1ab92ab55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e0cc563017646f6acea9127f7f525ee'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:13.026442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-459e3295-a5af-4828-a1c5-e9d1ab92ab55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e0cc563017646f6acea9127f7f525ee'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:13.027224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-459e3295-a5af-4828-a1c5-e9d1ab92ab55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e0cc563017646f6acea9127f7f525ee'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:13.027842 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-459e3295-a5af-4828-a1c5-e9d1ab92ab55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e0cc563017646f6acea9127f7f525ee'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:13.052974 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-459e3295-a5af-4828-a1c5-e9d1ab92ab55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e0cc563017646f6acea9127f7f525ee'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:13.124656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1142/2270] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:50:12 2022] DELETE /identity/v3/users/5e0cc563017646f6acea9127f7f525ee => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:50:13.577001 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172297e-120e-4cfa-a7ee-408a9876ac4b service placement] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:13.588972 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172297e-120e-4cfa-a7ee-408a9876ac4b service placement] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:13.657447 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b68b0-5318-4a3e-9461-163a1eebaf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:13.658311 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b68b0-5318-4a3e-9461-163a1eebaf12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:13.658610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b68b0-5318-4a3e-9461-163a1eebaf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:13.658882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b68b0-5318-4a3e-9461-163a1eebaf12 admin admin] PATH_INFO: `/v3/projects/eb2187f9489844ab8bbf1f69df83b2de` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:13.660109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69b68b0-5318-4a3e-9461-163a1eebaf12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb2187f9489844ab8bbf1f69df83b2de)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:13.662049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69b68b0-5318-4a3e-9461-163a1eebaf12 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:13.765518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d69b68b0-5318-4a3e-9461-163a1eebaf12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb2187f9489844ab8bbf1f69df83b2de', 'request_id': 'req-d69b68b0-5318-4a3e-9461-163a1eebaf12'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:13.771731 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d69b68b0-5318-4a3e-9461-163a1eebaf12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb2187f9489844ab8bbf1f69df83b2de', 'request_id': 'req-d69b68b0-5318-4a3e-9461-163a1eebaf12'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:13.847636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d69b68b0-5318-4a3e-9461-163a1eebaf12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb2187f9489844ab8bbf1f69df83b2de', 'request_id': 'req-d69b68b0-5318-4a3e-9461-163a1eebaf12'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:13.859432 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d69b68b0-5318-4a3e-9461-163a1eebaf12 admin admin] The token cache is being invalidate because project eb2187f9489844ab8bbf1f69df83b2de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:50:13.859720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d69b68b0-5318-4a3e-9461-163a1eebaf12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:13.860349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d69b68b0-5318-4a3e-9461-163a1eebaf12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:13.861907 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1129/2271] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:50:13 2022] DELETE /identity/v3/projects/eb2187f9489844ab8bbf1f69df83b2de => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:50:22.515655 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13ca92-08c6-4fc5-9944-8ec6ec5674b7 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:22.515655 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13ca92-08c6-4fc5-9944-8ec6ec5674b7 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:22.515655 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13ca92-08c6-4fc5-9944-8ec6ec5674b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:22.531939 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-0f13ca92-08c6-4fc5-9944-8ec6ec5674b7 None None] MFA Rules not processed for user `bcd5fc6167654972868941ed28c327b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:50:22.533115 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-0f13ca92-08c6-4fc5-9944-8ec6ec5674b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:22.585748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1143/2272] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:50:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:50:23.856292 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:50:28.087912 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b68b0-5318-4a3e-9461-163a1eebaf12 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:28.089447 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d69b68b0-5318-4a3e-9461-163a1eebaf12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:28.172987 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b68b0-5318-4a3e-9461-163a1eebaf12 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:28.178463 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889e71a-d16f-4268-b07c-4e19ba38a9e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:28.179215 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889e71a-d16f-4268-b07c-4e19ba38a9e6 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:28.179498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889e71a-d16f-4268-b07c-4e19ba38a9e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:28.179771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889e71a-d16f-4268-b07c-4e19ba38a9e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:28.181627 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3889e71a-d16f-4268-b07c-4e19ba38a9e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:28.195162 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889e71a-d16f-4268-b07c-4e19ba38a9e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:28.197003 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889e71a-d16f-4268-b07c-4e19ba38a9e6 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:28.208546 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1130/2273] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:50:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:50:30.015614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13ca92-08c6-4fc5-9944-8ec6ec5674b7 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:30.030911 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13ca92-08c6-4fc5-9944-8ec6ec5674b7 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:30.036616 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9249bc3-420b-4f10-8be0-620f0bb900d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:30.041260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9249bc3-420b-4f10-8be0-620f0bb900d4 service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:30.041260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9249bc3-420b-4f10-8be0-620f0bb900d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:30.041260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9249bc3-420b-4f10-8be0-620f0bb900d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:30.041260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c9249bc3-420b-4f10-8be0-620f0bb900d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:30.053587 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9249bc3-420b-4f10-8be0-620f0bb900d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:30.055324 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9249bc3-420b-4f10-8be0-620f0bb900d4 service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:30.060413 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:50:30.066732 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1144/2274] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:50:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:50:31.094655 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be8de7-9b93-4e74-ae63-708c606ce3f4 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:31.095587 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be8de7-9b93-4e74-ae63-708c606ce3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:31.096082 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be8de7-9b93-4e74-ae63-708c606ce3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:31.118393 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a4be8de7-9b93-4e74-ae63-708c606ce3f4 None None] MFA Rules not processed for user `030614ef56f143f8be6fcc0445ec9568`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:50:31.119995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a4be8de7-9b93-4e74-ae63-708c606ce3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:31.185179 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1131/2275] 10.222.0.13 () {58 vars in 967 bytes} [Thu Jun 23 11:50:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:50:31.208810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9249bc3-420b-4f10-8be0-620f0bb900d4 service nova] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:31.210288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c9249bc3-420b-4f10-8be0-620f0bb900d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:31.282808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9249bc3-420b-4f10-8be0-620f0bb900d4 service nova] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:31.287897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01af89-4c60-4640-ad53-32bd4c18209f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450b1fad96604fb09d0705895c6d3da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:31.289429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01af89-4c60-4640-ad53-32bd4c18209f service swift] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:31.290305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01af89-4c60-4640-ad53-32bd4c18209f service swift] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:31.290606 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01af89-4c60-4640-ad53-32bd4c18209f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:31.298963 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01af89-4c60-4640-ad53-32bd4c18209f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:31.300576 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01af89-4c60-4640-ad53-32bd4c18209f service swift] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:31.309238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1145/2276] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 23 11:50:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:50:34.170502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be8de7-9b93-4e74-ae63-708c606ce3f4 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:34.183921 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be8de7-9b93-4e74-ae63-708c606ce3f4 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:34.189351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c64877-a983-46fa-992b-7e483232951c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:34.189998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c64877-a983-46fa-992b-7e483232951c service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:34.190283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c64877-a983-46fa-992b-7e483232951c service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:34.190589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c64877-a983-46fa-992b-7e483232951c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:34.192345 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e0c64877-a983-46fa-992b-7e483232951c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:34.251441 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c64877-a983-46fa-992b-7e483232951c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:34.253051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c64877-a983-46fa-992b-7e483232951c service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:34.263091 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1132/2277] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:50:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:50:36.503452 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4f2b7-5ca4-429a-a363-165b9eb9f0ef None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:36.504320 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4f2b7-5ca4-429a-a363-165b9eb9f0ef None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:36.504604 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4f2b7-5ca4-429a-a363-165b9eb9f0ef None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:36.505985 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1146/2278] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 23 11:50:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:50:36.513104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b800b-e7d3-4d2c-a4e5-0673356287ac None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:36.513594 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b800b-e7d3-4d2c-a4e5-0673356287ac None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:36.513683 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b800b-e7d3-4d2c-a4e5-0673356287ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:36.541772 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-e18b800b-e7d3-4d2c-a4e5-0673356287ac None None] MFA Rules not processed for user `810fa3f54cd04f989252765f7e76e602`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:50:36.543037 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e18b800b-e7d3-4d2c-a4e5-0673356287ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:36.568853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:50:36.597047 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1133/2279] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 23 11:50:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:50:36.609320 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4f2b7-5ca4-429a-a363-165b9eb9f0ef None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:36.611071 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b7c4f2b7-5ca4-429a-a363-165b9eb9f0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:36.624535 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4f2b7-5ca4-429a-a363-165b9eb9f0ef None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:36.628544 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6b1d5-cb7f-4958-9252-58bbd6e639e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:36.629186 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6b1d5-cb7f-4958-9252-58bbd6e639e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:36.629488 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6b1d5-cb7f-4958-9252-58bbd6e639e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:36.629744 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6b1d5-cb7f-4958-9252-58bbd6e639e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:36.638331 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6b1d5-cb7f-4958-9252-58bbd6e639e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:36.639789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6b1d5-cb7f-4958-9252-58bbd6e639e2 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:36.647830 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1147/2280] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:50:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:50:36.731466 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637943c6-a5de-4827-8d63-b1e090ef6f89 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:36.731755 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637943c6-a5de-4827-8d63-b1e090ef6f89 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:36.732029 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637943c6-a5de-4827-8d63-b1e090ef6f89 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:36.733445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1134/2281] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 23 11:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:50:36.738640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8567a43-824c-4352-90a0-a7cdf2298dc5 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:36.738938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8567a43-824c-4352-90a0-a7cdf2298dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:36.739215 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8567a43-824c-4352-90a0-a7cdf2298dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:36.758220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c8567a43-824c-4352-90a0-a7cdf2298dc5 None None] MFA Rules not processed for user `161c5ed1e47e450290a1f428ee100b68`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:50:36.759497 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c8567a43-824c-4352-90a0-a7cdf2298dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:36.769145 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1148/2282] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 23 11:50:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:50:36.779445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637943c6-a5de-4827-8d63-b1e090ef6f89 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:36.791301 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637943c6-a5de-4827-8d63-b1e090ef6f89 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:36.794918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cccd4e-17cf-41d7-8837-bf9d5591baab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:36.795544 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cccd4e-17cf-41d7-8837-bf9d5591baab service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:36.795824 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cccd4e-17cf-41d7-8837-bf9d5591baab service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:36.796101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cccd4e-17cf-41d7-8837-bf9d5591baab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:36.807011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cccd4e-17cf-41d7-8837-bf9d5591baab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:36.808548 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cccd4e-17cf-41d7-8837-bf9d5591baab service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:36.819336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1135/2283] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:50:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:50:45.245941 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8567a43-824c-4352-90a0-a7cdf2298dc5 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:45.247422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c8567a43-824c-4352-90a0-a7cdf2298dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:45.260935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8567a43-824c-4352-90a0-a7cdf2298dc5 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:45.266283 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570798ff-6b33-421b-84d3-61e699118adf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:45.267007 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570798ff-6b33-421b-84d3-61e699118adf service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:45.267291 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570798ff-6b33-421b-84d3-61e699118adf service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:45.267566 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570798ff-6b33-421b-84d3-61e699118adf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:45.269254 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-570798ff-6b33-421b-84d3-61e699118adf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:45.331400 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570798ff-6b33-421b-84d3-61e699118adf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:45.333006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570798ff-6b33-421b-84d3-61e699118adf service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:45.342670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1149/2284] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:50:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:50:46.815027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=16 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:50:55.337777 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=70 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:51:04.774928 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfad7f-c8fa-4e1e-990b-c86df9e366c6 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:51:04.775474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfad7f-c8fa-4e1e-990b-c86df9e366c6 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:51:04.775580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfad7f-c8fa-4e1e-990b-c86df9e366c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:51:04.793232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-88cfad7f-c8fa-4e1e-990b-c86df9e366c6 None None] MFA Rules not processed for user `b7fd8323d29d445f903e2103453e8ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:51:04.795654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-88cfad7f-c8fa-4e1e-990b-c86df9e366c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:51:04.802596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1136/2285] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:51:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:51:14.788085 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:51:17.362418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612e609-b4cf-4685-b8e4-fbfbeb55cc12 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:51:17.363060 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612e609-b4cf-4685-b8e4-fbfbeb55cc12 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:51:17.363060 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612e609-b4cf-4685-b8e4-fbfbeb55cc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:51:17.380748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c612e609-b4cf-4685-b8e4-fbfbeb55cc12 None None] MFA Rules not processed for user `6cc5a4676f6a416cbdc8c67ebd80cdf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:51:17.381947 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c612e609-b4cf-4685-b8e4-fbfbeb55cc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:51:17.436177 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1150/2286] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:51:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:51:20.783612 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfad7f-c8fa-4e1e-990b-c86df9e366c6 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:51:20.804754 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfad7f-c8fa-4e1e-990b-c86df9e366c6 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:51:20.818588 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d8d34-3651-4009-b953-e6e4766a2876 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:51:20.819676 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d8d34-3651-4009-b953-e6e4766a2876 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:51:20.820116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d8d34-3651-4009-b953-e6e4766a2876 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:51:20.820556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d8d34-3651-4009-b953-e6e4766a2876 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:51:20.822573 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e33d8d34-3651-4009-b953-e6e4766a2876 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:51:20.844274 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d8d34-3651-4009-b953-e6e4766a2876 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:51:20.846678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d8d34-3651-4009-b953-e6e4766a2876 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:51:20.871501 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1137/2287] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:51:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:51:27.429258 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=25 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:51:30.854104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=6 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:51:41.219025 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612e609-b4cf-4685-b8e4-fbfbeb55cc12 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:51:41.232016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612e609-b4cf-4685-b8e4-fbfbeb55cc12 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:51:41.238123 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559a78f-b9ec-4034-a7f3-4ac8e85e654e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:51:41.238790 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559a78f-b9ec-4034-a7f3-4ac8e85e654e service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:51:41.239079 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559a78f-b9ec-4034-a7f3-4ac8e85e654e service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:51:41.239355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559a78f-b9ec-4034-a7f3-4ac8e85e654e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:51:41.241149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-3559a78f-b9ec-4034-a7f3-4ac8e85e654e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:51:41.297843 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3559a78f-b9ec-4034-a7f3-4ac8e85e654e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:51:41.299481 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3559a78f-b9ec-4034-a7f3-4ac8e85e654e service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:51:41.308294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1151/2288] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:51:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:51:51.304215 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=28 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:51:54.112867 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18017c1-b085-4c0a-92b5-2e6250729326 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:51:54.113364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18017c1-b085-4c0a-92b5-2e6250729326 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:51:54.113459 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18017c1-b085-4c0a-92b5-2e6250729326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:51:54.131438 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d18017c1-b085-4c0a-92b5-2e6250729326 None None] MFA Rules not processed for user `41cb2b9a5156470aaeca9cc10f402ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:51:54.140255 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d18017c1-b085-4c0a-92b5-2e6250729326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:51:54.140720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1138/2289] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:51:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:51:54.483875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559a78f-b9ec-4034-a7f3-4ac8e85e654e service nova] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:51:54.485597 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-3559a78f-b9ec-4034-a7f3-4ac8e85e654e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:51:54.498223 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559a78f-b9ec-4034-a7f3-4ac8e85e654e service nova] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:51:54.502918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c21e26-2d93-4d9c-a6d2-4ec6270ce3d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:51:54.503561 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c21e26-2d93-4d9c-a6d2-4ec6270ce3d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:51:54.509462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c21e26-2d93-4d9c-a6d2-4ec6270ce3d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:51:54.509462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c21e26-2d93-4d9c-a6d2-4ec6270ce3d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:51:54.509462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-57c21e26-2d93-4d9c-a6d2-4ec6270ce3d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:51:54.515491 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c21e26-2d93-4d9c-a6d2-4ec6270ce3d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:51:54.517051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c21e26-2d93-4d9c-a6d2-4ec6270ce3d4 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:51:54.525018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1152/2290] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:51:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:51:59.548319 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18017c1-b085-4c0a-92b5-2e6250729326 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:51:59.558584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18017c1-b085-4c0a-92b5-2e6250729326 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:51:59.564969 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957183e-ce02-4244-858e-762bf3911241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:51:59.564969 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957183e-ce02-4244-858e-762bf3911241 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:51:59.564969 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957183e-ce02-4244-858e-762bf3911241 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:51:59.565268 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957183e-ce02-4244-858e-762bf3911241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:51:59.567151 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c957183e-ce02-4244-858e-762bf3911241 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:51:59.645751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957183e-ce02-4244-858e-762bf3911241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:51:59.647312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957183e-ce02-4244-858e-762bf3911241 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:51:59.659201 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1139/2291] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:51:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:52:03.979182 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c21e26-2d93-4d9c-a6d2-4ec6270ce3d4 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:03.980761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-57c21e26-2d93-4d9c-a6d2-4ec6270ce3d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:03.996092 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c21e26-2d93-4d9c-a6d2-4ec6270ce3d4 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:04.001878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a2111-3dfe-4a12-8344-0f639c1b72d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:04.002986 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a2111-3dfe-4a12-8344-0f639c1b72d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:04.003224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a2111-3dfe-4a12-8344-0f639c1b72d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:04.003567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a2111-3dfe-4a12-8344-0f639c1b72d3 admin admin] PATH_INFO: `/v3/users/bcd5fc6167654972868941ed28c327b6` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:04.004786 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a2111-3dfe-4a12-8344-0f639c1b72d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcd5fc6167654972868941ed28c327b6)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:04.006923 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a2111-3dfe-4a12-8344-0f639c1b72d3 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:04.243027 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6c3a2111-3dfe-4a12-8344-0f639c1b72d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcd5fc6167654972868941ed28c327b6'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:04.250413 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6c3a2111-3dfe-4a12-8344-0f639c1b72d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcd5fc6167654972868941ed28c327b6'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:04.251103 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6c3a2111-3dfe-4a12-8344-0f639c1b72d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcd5fc6167654972868941ed28c327b6'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:04.251711 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6c3a2111-3dfe-4a12-8344-0f639c1b72d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcd5fc6167654972868941ed28c327b6'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:04.271375 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6c3a2111-3dfe-4a12-8344-0f639c1b72d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcd5fc6167654972868941ed28c327b6'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:04.362887 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1153/2292] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:52:03 2022] DELETE /identity/v3/users/bcd5fc6167654972868941ed28c327b6 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:04.701352 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957183e-ce02-4244-858e-762bf3911241 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:04.712079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957183e-ce02-4244-858e-762bf3911241 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:04.782140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000552b-87b0-4ad1-be4d-d28ccba31334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:04.783051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000552b-87b0-4ad1-be4d-d28ccba31334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:04.783341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000552b-87b0-4ad1-be4d-d28ccba31334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:04.783622 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000552b-87b0-4ad1-be4d-d28ccba31334 admin admin] PATH_INFO: `/v3/projects/b6241924efa944a68503a87b695c7410` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:04.784832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8000552b-87b0-4ad1-be4d-d28ccba31334 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6241924efa944a68503a87b695c7410)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:04.786806 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8000552b-87b0-4ad1-be4d-d28ccba31334 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:04.903816 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8000552b-87b0-4ad1-be4d-d28ccba31334 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6241924efa944a68503a87b695c7410', 'request_id': 'req-8000552b-87b0-4ad1-be4d-d28ccba31334'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:04.915212 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8000552b-87b0-4ad1-be4d-d28ccba31334 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6241924efa944a68503a87b695c7410', 'request_id': 'req-8000552b-87b0-4ad1-be4d-d28ccba31334'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:04.982918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8000552b-87b0-4ad1-be4d-d28ccba31334 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6241924efa944a68503a87b695c7410', 'request_id': 'req-8000552b-87b0-4ad1-be4d-d28ccba31334'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:04.994503 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8000552b-87b0-4ad1-be4d-d28ccba31334 admin admin] The token cache is being invalidate because project b6241924efa944a68503a87b695c7410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:52:04.994832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8000552b-87b0-4ad1-be4d-d28ccba31334 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:04.995456 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8000552b-87b0-4ad1-be4d-d28ccba31334 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:04.996997 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1140/2293] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:52:04 2022] DELETE /identity/v3/projects/b6241924efa944a68503a87b695c7410 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:05.002117 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901166b-ad19-4157-afc0-34a09da1a92e None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:05.002562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901166b-ad19-4157-afc0-34a09da1a92e None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:05.002672 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901166b-ad19-4157-afc0-34a09da1a92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:05.021683 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-a901166b-ad19-4157-afc0-34a09da1a92e None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:05.022911 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a901166b-ad19-4157-afc0-34a09da1a92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:05.089989 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1154/2294] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:05.099876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdef65-cb04-4ee6-993d-050d34600154 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:05.100168 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdef65-cb04-4ee6-993d-050d34600154 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:05.100445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdef65-cb04-4ee6-993d-050d34600154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:05.119657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-77cdef65-cb04-4ee6-993d-050d34600154 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:05.120996 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-77cdef65-cb04-4ee6-993d-050d34600154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:05.129946 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1141/2295] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:05.134377 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901166b-ad19-4157-afc0-34a09da1a92e None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:05.145272 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901166b-ad19-4157-afc0-34a09da1a92e None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:05.150256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13826f-214c-4fa3-b142-caad0e1e5cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:05.151095 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13826f-214c-4fa3-b142-caad0e1e5cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:05.151563 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13826f-214c-4fa3-b142-caad0e1e5cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:05.151847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13826f-214c-4fa3-b142-caad0e1e5cd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:05.152649 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13826f-214c-4fa3-b142-caad0e1e5cd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:52:05.154703 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13826f-214c-4fa3-b142-caad0e1e5cd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:05.154892 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13826f-214c-4fa3-b142-caad0e1e5cd8 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:05.168247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1155/2296] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:52:05.172679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdef65-cb04-4ee6-993d-050d34600154 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:05.187609 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdef65-cb04-4ee6-993d-050d34600154 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:05.194429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111f0c4-993d-4249-8ee9-6b0d9deffaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:05.195162 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111f0c4-993d-4249-8ee9-6b0d9deffaec admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:05.195574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111f0c4-993d-4249-8ee9-6b0d9deffaec admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:05.195753 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111f0c4-993d-4249-8ee9-6b0d9deffaec admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:05.196596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111f0c4-993d-4249-8ee9-6b0d9deffaec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:05.198420 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111f0c4-993d-4249-8ee9-6b0d9deffaec admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:05.207788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:05.314059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1142/2297] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:52:05 2022] POST /identity/v3/projects => generated 401 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:52:05.318111 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13826f-214c-4fa3-b142-caad0e1e5cd8 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:05.329253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13826f-214c-4fa3-b142-caad0e1e5cd8 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:05.372985 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:05.385593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ddbfd-7330-4139-a1be-4e763f816a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:05.386300 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ddbfd-7330-4139-a1be-4e763f816a52 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:05.386614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ddbfd-7330-4139-a1be-4e763f816a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:05.386899 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ddbfd-7330-4139-a1be-4e763f816a52 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:05.387591 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ddbfd-7330-4139-a1be-4e763f816a52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:05.390201 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ddbfd-7330-4139-a1be-4e763f816a52 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:05.481735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1156/2298] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:52:05 2022] POST /identity/v3/users => generated 301 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:52:05.486340 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111f0c4-993d-4249-8ee9-6b0d9deffaec admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:05.497079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111f0c4-993d-4249-8ee9-6b0d9deffaec admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:05.502605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0c6f4-e5fa-4de0-9f84-72514a7ad7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:05.503193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0c6f4-e5fa-4de0-9f84-72514a7ad7e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:05.503478 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0c6f4-e5fa-4de0-9f84-72514a7ad7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:05.503759 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0c6f4-e5fa-4de0-9f84-72514a7ad7e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:05.504483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0c6f4-e5fa-4de0-9f84-72514a7ad7e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:05.506080 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0c6f4-e5fa-4de0-9f84-72514a7ad7e1 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:05.516880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1143/2299] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:05.520954 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ddbfd-7330-4139-a1be-4e763f816a52 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:05.532704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ddbfd-7330-4139-a1be-4e763f816a52 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:05.535884 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb74134-41d3-4172-972d-39882f1714f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:05.536538 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb74134-41d3-4172-972d-39882f1714f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:05.536822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb74134-41d3-4172-972d-39882f1714f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:05.537096 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb74134-41d3-4172-972d-39882f1714f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:05.537882 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb74134-41d3-4172-972d-39882f1714f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:05.539526 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb74134-41d3-4172-972d-39882f1714f3 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:05.550772 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1157/2300] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:05.554836 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0c6f4-e5fa-4de0-9f84-72514a7ad7e1 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:05.567026 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0c6f4-e5fa-4de0-9f84-72514a7ad7e1 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:05.571729 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd517f7-40b8-4990-88d9-5c80baf9ee12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:05.572655 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd517f7-40b8-4990-88d9-5c80baf9ee12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:05.573082 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd517f7-40b8-4990-88d9-5c80baf9ee12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:05.573367 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd517f7-40b8-4990-88d9-5c80baf9ee12 admin admin] PATH_INFO: `/v3/projects/6bc9b153de8e4cb1854e1fcc17595df7/users/4fda906390f74aa3a7bb8c81eec1bf79/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:05.588557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd517f7-40b8-4990-88d9-5c80baf9ee12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bc9b153de8e4cb1854e1fcc17595df7, user_id=4fda906390f74aa3a7bb8c81eec1bf79, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:05.591398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd517f7-40b8-4990-88d9-5c80baf9ee12 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:05.665418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1144/2301] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:52:05 2022] PUT /identity/v3/projects/6bc9b153de8e4cb1854e1fcc17595df7/users/4fda906390f74aa3a7bb8c81eec1bf79/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:05.675957 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a25c26-23e6-48d4-b29a-fee64adde780 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:05.676390 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a25c26-23e6-48d4-b29a-fee64adde780 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:05.676688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a25c26-23e6-48d4-b29a-fee64adde780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:05.706160 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-d4a25c26-23e6-48d4-b29a-fee64adde780 None None] MFA Rules not processed for user `4fda906390f74aa3a7bb8c81eec1bf79`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:05.707465 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d4a25c26-23e6-48d4-b29a-fee64adde780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:05.777149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1158/2302] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:05.782223 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd517f7-40b8-4990-88d9-5c80baf9ee12 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:05.850773 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd517f7-40b8-4990-88d9-5c80baf9ee12 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:05.855550 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04d6ed-9e6e-4a2d-a7a6-2e85e4d8ae69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:05.856256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04d6ed-9e6e-4a2d-a7a6-2e85e4d8ae69 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:05.856700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04d6ed-9e6e-4a2d-a7a6-2e85e4d8ae69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:05.856988 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04d6ed-9e6e-4a2d-a7a6-2e85e4d8ae69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:05.857684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04d6ed-9e6e-4a2d-a7a6-2e85e4d8ae69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:05.860228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04d6ed-9e6e-4a2d-a7a6-2e85e4d8ae69 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:05.968261 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1145/2303] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:52:05 2022] POST /identity/v3/projects => generated 403 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:52:05.972994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a25c26-23e6-48d4-b29a-fee64adde780 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:05.984338 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a25c26-23e6-48d4-b29a-fee64adde780 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:06.039350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f89921-e618-44ec-83c4-063a1eeaa9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:06.040038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f89921-e618-44ec-83c4-063a1eeaa9bf admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:06.040267 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f89921-e618-44ec-83c4-063a1eeaa9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:06.040545 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f89921-e618-44ec-83c4-063a1eeaa9bf admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:06.041244 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f89921-e618-44ec-83c4-063a1eeaa9bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:06.043255 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f89921-e618-44ec-83c4-063a1eeaa9bf admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:06.143430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1159/2304] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:52:05 2022] POST /identity/v3/users => generated 308 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:52:06.147343 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04d6ed-9e6e-4a2d-a7a6-2e85e4d8ae69 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:06.158624 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04d6ed-9e6e-4a2d-a7a6-2e85e4d8ae69 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:06.163720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2b1ce-935e-4bef-9cd3-d792bdcaac69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:06.164390 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2b1ce-935e-4bef-9cd3-d792bdcaac69 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:06.164786 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2b1ce-935e-4bef-9cd3-d792bdcaac69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:06.165064 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2b1ce-935e-4bef-9cd3-d792bdcaac69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:06.165918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2b1ce-935e-4bef-9cd3-d792bdcaac69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:06.167593 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2b1ce-935e-4bef-9cd3-d792bdcaac69 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:06.179489 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1146/2305] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:06.185706 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f89921-e618-44ec-83c4-063a1eeaa9bf admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:06.196214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f89921-e618-44ec-83c4-063a1eeaa9bf admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:06.200839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a0e6d-58bf-4195-9990-f36c379465e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:06.201709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a0e6d-58bf-4195-9990-f36c379465e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:06.202025 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a0e6d-58bf-4195-9990-f36c379465e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:06.202306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a0e6d-58bf-4195-9990-f36c379465e1 admin admin] PATH_INFO: `/v3/projects/15608b0d27144c57a7f4b6d011026fb8/users/36b29301e7dd44da9cc7ba9fdc7eb4ee/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:06.216571 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a0e6d-58bf-4195-9990-f36c379465e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15608b0d27144c57a7f4b6d011026fb8, user_id=36b29301e7dd44da9cc7ba9fdc7eb4ee, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:06.219850 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a0e6d-58bf-4195-9990-f36c379465e1 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:06.294648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1160/2306] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:52:06 2022] PUT /identity/v3/projects/15608b0d27144c57a7f4b6d011026fb8/users/36b29301e7dd44da9cc7ba9fdc7eb4ee/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:06.298664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2b1ce-935e-4bef-9cd3-d792bdcaac69 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:06.308762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2b1ce-935e-4bef-9cd3-d792bdcaac69 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:06.368229 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f537107-a71f-4e9c-9ec5-cbea49663575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:06.368905 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f537107-a71f-4e9c-9ec5-cbea49663575 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:06.369192 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f537107-a71f-4e9c-9ec5-cbea49663575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:06.369472 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f537107-a71f-4e9c-9ec5-cbea49663575 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:06.370226 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f537107-a71f-4e9c-9ec5-cbea49663575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:06.371888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f537107-a71f-4e9c-9ec5-cbea49663575 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:06.382199 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1147/2307] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:06.386585 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a0e6d-58bf-4195-9990-f36c379465e1 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:06.404103 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a0e6d-58bf-4195-9990-f36c379465e1 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:06.404103 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b076f75-3baa-4a38-8cdc-77c548935360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:06.404103 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b076f75-3baa-4a38-8cdc-77c548935360 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:06.404103 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b076f75-3baa-4a38-8cdc-77c548935360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:06.404718 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b076f75-3baa-4a38-8cdc-77c548935360 admin admin] PATH_INFO: `/v3/projects/15608b0d27144c57a7f4b6d011026fb8/users/36b29301e7dd44da9cc7ba9fdc7eb4ee/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:06.406955 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b076f75-3baa-4a38-8cdc-77c548935360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15608b0d27144c57a7f4b6d011026fb8, user_id=36b29301e7dd44da9cc7ba9fdc7eb4ee, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:06.409692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b076f75-3baa-4a38-8cdc-77c548935360 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:06.433198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1161/2308] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:52:06 2022] PUT /identity/v3/projects/15608b0d27144c57a7f4b6d011026fb8/users/36b29301e7dd44da9cc7ba9fdc7eb4ee/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:06.441186 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e627e-e7d3-4eb9-a0ef-7476bebd874e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:06.441186 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e627e-e7d3-4eb9-a0ef-7476bebd874e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:06.441186 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e627e-e7d3-4eb9-a0ef-7476bebd874e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:06.465183 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-454e627e-e7d3-4eb9-a0ef-7476bebd874e None None] MFA Rules not processed for user `36b29301e7dd44da9cc7ba9fdc7eb4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:06.466573 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-454e627e-e7d3-4eb9-a0ef-7476bebd874e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:06.472187 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:06.543692 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1148/2309] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:06 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:06.555047 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b076f75-3baa-4a38-8cdc-77c548935360 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:06.561484 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-2b076f75-3baa-4a38-8cdc-77c548935360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:06.648584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:06.664497 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b076f75-3baa-4a38-8cdc-77c548935360 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:06.667909 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958021ed-276e-46b2-a73a-894e6ed3bc02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:06.668552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958021ed-276e-46b2-a73a-894e6ed3bc02 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:06.668834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958021ed-276e-46b2-a73a-894e6ed3bc02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:06.669148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958021ed-276e-46b2-a73a-894e6ed3bc02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:06.680088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958021ed-276e-46b2-a73a-894e6ed3bc02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:06.681689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958021ed-276e-46b2-a73a-894e6ed3bc02 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:06.693336 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1162/2310] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:06 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:52:06.851283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e627e-e7d3-4eb9-a0ef-7476bebd874e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:06.926794 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e627e-e7d3-4eb9-a0ef-7476bebd874e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:06.931957 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b60dc4-e28b-4bfe-9360-3d0235708d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:06.932878 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b60dc4-e28b-4bfe-9360-3d0235708d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:06.933165 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b60dc4-e28b-4bfe-9360-3d0235708d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:06.933452 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b60dc4-e28b-4bfe-9360-3d0235708d6d admin admin] PATH_INFO: `/v3/users/4fda906390f74aa3a7bb8c81eec1bf79` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:06.934753 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b60dc4-e28b-4bfe-9360-3d0235708d6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fda906390f74aa3a7bb8c81eec1bf79)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:06.937333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b60dc4-e28b-4bfe-9360-3d0235708d6d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:07.136887 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e0b60dc4-e28b-4bfe-9360-3d0235708d6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fda906390f74aa3a7bb8c81eec1bf79'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:07.144227 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e0b60dc4-e28b-4bfe-9360-3d0235708d6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fda906390f74aa3a7bb8c81eec1bf79'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:07.145081 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e0b60dc4-e28b-4bfe-9360-3d0235708d6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fda906390f74aa3a7bb8c81eec1bf79'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:07.145724 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e0b60dc4-e28b-4bfe-9360-3d0235708d6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fda906390f74aa3a7bb8c81eec1bf79'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:07.158636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e0b60dc4-e28b-4bfe-9360-3d0235708d6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fda906390f74aa3a7bb8c81eec1bf79'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:07.246268 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1149/2311] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:52:06 2022] DELETE /identity/v3/users/4fda906390f74aa3a7bb8c81eec1bf79 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:07.252367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ad144-76f9-46d0-98e3-de6b657b5225 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:07.252882 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ad144-76f9-46d0-98e3-de6b657b5225 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:07.253011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ad144-76f9-46d0-98e3-de6b657b5225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:07.271876 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-eb1ad144-76f9-46d0-98e3-de6b657b5225 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:07.273230 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-eb1ad144-76f9-46d0-98e3-de6b657b5225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:07.337651 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1163/2312] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:07.348035 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b60dc4-e28b-4bfe-9360-3d0235708d6d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:07.349488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e0b60dc4-e28b-4bfe-9360-3d0235708d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:07.382033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:07.413676 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b60dc4-e28b-4bfe-9360-3d0235708d6d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:07.417302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb7c3f-2acb-4fe4-947c-9b1d5ada4c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:07.417951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb7c3f-2acb-4fe4-947c-9b1d5ada4c88 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:07.418283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb7c3f-2acb-4fe4-947c-9b1d5ada4c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:07.418575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb7c3f-2acb-4fe4-947c-9b1d5ada4c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:07.429760 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb7c3f-2acb-4fe4-947c-9b1d5ada4c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:07.431351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb7c3f-2acb-4fe4-947c-9b1d5ada4c88 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:07.442028 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1150/2313] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:52:07.636364 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6dc143-a7ea-4915-a406-d02729cbc7fd None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:07.636657 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6dc143-a7ea-4915-a406-d02729cbc7fd None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:07.636937 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6dc143-a7ea-4915-a406-d02729cbc7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:07.654545 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-bb6dc143-a7ea-4915-a406-d02729cbc7fd None None] MFA Rules not processed for user `d1b988f3241a46308b7c3683e2943b78`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:07.657875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-bb6dc143-a7ea-4915-a406-d02729cbc7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:07.719214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1164/2314] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:07.894186 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb7c3f-2acb-4fe4-947c-9b1d5ada4c88 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:07.896228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-8afb7c3f-2acb-4fe4-947c-9b1d5ada4c88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:07.910977 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb7c3f-2acb-4fe4-947c-9b1d5ada4c88 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:07.916561 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70829fa-06b4-4375-9d6d-515371a18e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:07.919308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70829fa-06b4-4375-9d6d-515371a18e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:07.919308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70829fa-06b4-4375-9d6d-515371a18e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:07.919308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70829fa-06b4-4375-9d6d-515371a18e1a admin admin] PATH_INFO: `/v3/projects/6bc9b153de8e4cb1854e1fcc17595df7` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:07.920116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70829fa-06b4-4375-9d6d-515371a18e1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bc9b153de8e4cb1854e1fcc17595df7)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:07.922325 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70829fa-06b4-4375-9d6d-515371a18e1a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:08.069210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d70829fa-06b4-4375-9d6d-515371a18e1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bc9b153de8e4cb1854e1fcc17595df7', 'request_id': 'req-d70829fa-06b4-4375-9d6d-515371a18e1a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:08.076243 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d70829fa-06b4-4375-9d6d-515371a18e1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bc9b153de8e4cb1854e1fcc17595df7', 'request_id': 'req-d70829fa-06b4-4375-9d6d-515371a18e1a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:08.162412 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d70829fa-06b4-4375-9d6d-515371a18e1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bc9b153de8e4cb1854e1fcc17595df7', 'request_id': 'req-d70829fa-06b4-4375-9d6d-515371a18e1a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:08.174242 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d70829fa-06b4-4375-9d6d-515371a18e1a admin admin] The token cache is being invalidate because project 6bc9b153de8e4cb1854e1fcc17595df7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:52:08.174884 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d70829fa-06b4-4375-9d6d-515371a18e1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:08.175607 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d70829fa-06b4-4375-9d6d-515371a18e1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:08.177358 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1151/2315] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:52:07 2022] DELETE /identity/v3/projects/6bc9b153de8e4cb1854e1fcc17595df7 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:08.181366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6dc143-a7ea-4915-a406-d02729cbc7fd None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:08.193569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6dc143-a7ea-4915-a406-d02729cbc7fd None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:08.229552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:08.255618 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f04217-2ee5-424c-a2fa-f5dc4eb89f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:08.256497 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f04217-2ee5-424c-a2fa-f5dc4eb89f53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:08.256832 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f04217-2ee5-424c-a2fa-f5dc4eb89f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:08.257112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f04217-2ee5-424c-a2fa-f5dc4eb89f53 admin admin] PATH_INFO: `/v3/users/36b29301e7dd44da9cc7ba9fdc7eb4ee` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:08.258419 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f04217-2ee5-424c-a2fa-f5dc4eb89f53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36b29301e7dd44da9cc7ba9fdc7eb4ee)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:08.260761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f04217-2ee5-424c-a2fa-f5dc4eb89f53 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:08.497060 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c2f04217-2ee5-424c-a2fa-f5dc4eb89f53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36b29301e7dd44da9cc7ba9fdc7eb4ee'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:08.504293 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c2f04217-2ee5-424c-a2fa-f5dc4eb89f53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36b29301e7dd44da9cc7ba9fdc7eb4ee'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:08.505049 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c2f04217-2ee5-424c-a2fa-f5dc4eb89f53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36b29301e7dd44da9cc7ba9fdc7eb4ee'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:08.505675 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c2f04217-2ee5-424c-a2fa-f5dc4eb89f53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36b29301e7dd44da9cc7ba9fdc7eb4ee'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:08.517601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c2f04217-2ee5-424c-a2fa-f5dc4eb89f53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36b29301e7dd44da9cc7ba9fdc7eb4ee'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:08.605119 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1165/2316] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:52:08 2022] DELETE /identity/v3/users/36b29301e7dd44da9cc7ba9fdc7eb4ee => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:09.065265 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70829fa-06b4-4375-9d6d-515371a18e1a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:09.066841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d70829fa-06b4-4375-9d6d-515371a18e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:09.081104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70829fa-06b4-4375-9d6d-515371a18e1a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:09.144911 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7fe2b-bff1-4da0-acd8-8b2d4c806779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:09.145784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7fe2b-bff1-4da0-acd8-8b2d4c806779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:09.146067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7fe2b-bff1-4da0-acd8-8b2d4c806779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:09.146345 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7fe2b-bff1-4da0-acd8-8b2d4c806779 admin admin] PATH_INFO: `/v3/projects/15608b0d27144c57a7f4b6d011026fb8` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:09.147616 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7fe2b-bff1-4da0-acd8-8b2d4c806779 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15608b0d27144c57a7f4b6d011026fb8)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:09.149583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7fe2b-bff1-4da0-acd8-8b2d4c806779 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:09.290645 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:09.291887 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f8b7fe2b-bff1-4da0-acd8-8b2d4c806779 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15608b0d27144c57a7f4b6d011026fb8', 'request_id': 'req-f8b7fe2b-bff1-4da0-acd8-8b2d4c806779'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:09.298579 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f8b7fe2b-bff1-4da0-acd8-8b2d4c806779 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15608b0d27144c57a7f4b6d011026fb8', 'request_id': 'req-f8b7fe2b-bff1-4da0-acd8-8b2d4c806779'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:09.411331 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f8b7fe2b-bff1-4da0-acd8-8b2d4c806779 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15608b0d27144c57a7f4b6d011026fb8', 'request_id': 'req-f8b7fe2b-bff1-4da0-acd8-8b2d4c806779'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:09.423066 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f8b7fe2b-bff1-4da0-acd8-8b2d4c806779 admin admin] The token cache is being invalidate because project 15608b0d27144c57a7f4b6d011026fb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:52:09.423361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f8b7fe2b-bff1-4da0-acd8-8b2d4c806779 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:09.424086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f8b7fe2b-bff1-4da0-acd8-8b2d4c806779 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:09.425668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1152/2317] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:52:09 2022] DELETE /identity/v3/projects/15608b0d27144c57a7f4b6d011026fb8 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:09.430805 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f59fe-02c3-42cd-8f53-f1da16ff56c5 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:09.431268 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f59fe-02c3-42cd-8f53-f1da16ff56c5 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:09.431507 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f59fe-02c3-42cd-8f53-f1da16ff56c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:09.450603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-3b6f59fe-02c3-42cd-8f53-f1da16ff56c5 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:09.451866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-3b6f59fe-02c3-42cd-8f53-f1da16ff56c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:09.525622 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1166/2318] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:09.533843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a941836-c929-4791-a443-5a7db5d5f372 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:09.534129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a941836-c929-4791-a443-5a7db5d5f372 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:09.534404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a941836-c929-4791-a443-5a7db5d5f372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:09.554152 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3a941836-c929-4791-a443-5a7db5d5f372 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:09.555590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3a941836-c929-4791-a443-5a7db5d5f372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:09.565008 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1153/2319] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:09.569272 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f59fe-02c3-42cd-8f53-f1da16ff56c5 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:09.580565 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f59fe-02c3-42cd-8f53-f1da16ff56c5 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:09.585577 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281879a-42a6-406a-861b-4f98b50c7566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:09.586764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281879a-42a6-406a-861b-4f98b50c7566 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:09.587232 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281879a-42a6-406a-861b-4f98b50c7566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:09.587703 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281879a-42a6-406a-861b-4f98b50c7566 admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:09.588609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281879a-42a6-406a-861b-4f98b50c7566 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:52:09.589095 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281879a-42a6-406a-861b-4f98b50c7566 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:09.596456 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281879a-42a6-406a-861b-4f98b50c7566 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:09.610208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1167/2320] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:52:09.615047 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a941836-c929-4791-a443-5a7db5d5f372 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:09.624678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a941836-c929-4791-a443-5a7db5d5f372 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:09.629676 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de258b-ff90-4e45-a5bf-36c4c83c2ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:09.630345 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de258b-ff90-4e45-a5bf-36c4c83c2ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:09.630672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de258b-ff90-4e45-a5bf-36c4c83c2ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:09.630952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de258b-ff90-4e45-a5bf-36c4c83c2ff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:09.631648 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de258b-ff90-4e45-a5bf-36c4c83c2ff7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:09.633663 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de258b-ff90-4e45-a5bf-36c4c83c2ff7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:09.715643 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1154/2321] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:52:09 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:52:09.719913 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281879a-42a6-406a-861b-4f98b50c7566 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:09.730149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281879a-42a6-406a-861b-4f98b50c7566 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:09.774431 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:09.789867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d23d8-50b2-450e-8561-755540c6e078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:09.790558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d23d8-50b2-450e-8561-755540c6e078 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:09.790845 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d23d8-50b2-450e-8561-755540c6e078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:09.791119 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d23d8-50b2-450e-8561-755540c6e078 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:09.791815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d23d8-50b2-450e-8561-755540c6e078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:09.793735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d23d8-50b2-450e-8561-755540c6e078 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:09.901692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1168/2322] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:52:09 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:52:09.902859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de258b-ff90-4e45-a5bf-36c4c83c2ff7 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:09.919083 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de258b-ff90-4e45-a5bf-36c4c83c2ff7 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:09.924942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c6408-a849-4f3f-9835-6e467b5a5323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:09.925859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c6408-a849-4f3f-9835-6e467b5a5323 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:09.926150 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c6408-a849-4f3f-9835-6e467b5a5323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:09.926425 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c6408-a849-4f3f-9835-6e467b5a5323 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:09.927275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c6408-a849-4f3f-9835-6e467b5a5323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:09.929143 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c6408-a849-4f3f-9835-6e467b5a5323 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:09.943495 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1155/2323] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:09.947839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d23d8-50b2-450e-8561-755540c6e078 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:09.959202 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d23d8-50b2-450e-8561-755540c6e078 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:09.964026 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171df70-c69e-47f5-b782-8aaf0bb501b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:09.964682 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171df70-c69e-47f5-b782-8aaf0bb501b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:09.964963 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171df70-c69e-47f5-b782-8aaf0bb501b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:09.968324 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171df70-c69e-47f5-b782-8aaf0bb501b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:09.968324 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171df70-c69e-47f5-b782-8aaf0bb501b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:09.968324 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171df70-c69e-47f5-b782-8aaf0bb501b3 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:09.979383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1169/2324] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:52:09.984054 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c6408-a849-4f3f-9835-6e467b5a5323 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:09.994210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c6408-a849-4f3f-9835-6e467b5a5323 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:10.000623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdd050-f4bf-4608-adbf-a36bbb0a3840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:10.001522 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdd050-f4bf-4608-adbf-a36bbb0a3840 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:10.001806 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdd050-f4bf-4608-adbf-a36bbb0a3840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:10.002650 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdd050-f4bf-4608-adbf-a36bbb0a3840 admin admin] PATH_INFO: `/v3/projects/a7ac39b34db144abbca29cf69555e86b/users/f502f724cd4149a89af06a76c2b11db9/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:10.020640 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abdd050-f4bf-4608-adbf-a36bbb0a3840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7ac39b34db144abbca29cf69555e86b, user_id=f502f724cd4149a89af06a76c2b11db9, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:10.024295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abdd050-f4bf-4608-adbf-a36bbb0a3840 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:10.110248 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1156/2325] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:52:09 2022] PUT /identity/v3/projects/a7ac39b34db144abbca29cf69555e86b/users/f502f724cd4149a89af06a76c2b11db9/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:52:10.120420 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bde3e4-5571-44bc-810a-4e51cf3acde0 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:10.120705 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bde3e4-5571-44bc-810a-4e51cf3acde0 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:10.121007 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bde3e4-5571-44bc-810a-4e51cf3acde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:10.149119 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-e7bde3e4-5571-44bc-810a-4e51cf3acde0 None None] MFA Rules not processed for user `f502f724cd4149a89af06a76c2b11db9`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:10.150391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e7bde3e4-5571-44bc-810a-4e51cf3acde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:10.215088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1170/2326] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:10.219621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdd050-f4bf-4608-adbf-a36bbb0a3840 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:10.283910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdd050-f4bf-4608-adbf-a36bbb0a3840 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:10.288746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a89201-80fb-42bf-bab2-787af7ee253f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:10.296258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a89201-80fb-42bf-bab2-787af7ee253f admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:10.296258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a89201-80fb-42bf-bab2-787af7ee253f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:10.296258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a89201-80fb-42bf-bab2-787af7ee253f admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:10.296258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a89201-80fb-42bf-bab2-787af7ee253f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:10.296258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a89201-80fb-42bf-bab2-787af7ee253f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:10.393755 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1157/2327] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:52:10 2022] POST /identity/v3/projects => generated 379 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:52:10.394923 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bde3e4-5571-44bc-810a-4e51cf3acde0 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:10.406633 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bde3e4-5571-44bc-810a-4e51cf3acde0 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:10.471452 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad4ebd-ba0f-427f-be82-7ae0960a16e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:10.472189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad4ebd-ba0f-427f-be82-7ae0960a16e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:10.472472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad4ebd-ba0f-427f-be82-7ae0960a16e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:10.473000 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad4ebd-ba0f-427f-be82-7ae0960a16e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:10.473723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad4ebd-ba0f-427f-be82-7ae0960a16e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:10.475920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad4ebd-ba0f-427f-be82-7ae0960a16e1 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:10.569877 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1171/2328] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:52:10 2022] POST /identity/v3/users => generated 296 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:52:10.572166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a89201-80fb-42bf-bab2-787af7ee253f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:10.586949 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a89201-80fb-42bf-bab2-787af7ee253f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:10.594457 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ec6fd-c019-4ce3-b1cb-da4e7e7b2bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:10.595544 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ec6fd-c019-4ce3-b1cb-da4e7e7b2bea admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:10.595894 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ec6fd-c019-4ce3-b1cb-da4e7e7b2bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:10.596230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ec6fd-c019-4ce3-b1cb-da4e7e7b2bea admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:10.597124 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6ec6fd-c019-4ce3-b1cb-da4e7e7b2bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:10.599265 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6ec6fd-c019-4ce3-b1cb-da4e7e7b2bea admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:10.614431 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1158/2329] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:10.618249 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad4ebd-ba0f-427f-be82-7ae0960a16e1 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:10.631611 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad4ebd-ba0f-427f-be82-7ae0960a16e1 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:10.638157 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f300914-bf9d-412d-a734-e5b2b969bf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:10.639542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f300914-bf9d-412d-a734-e5b2b969bf77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:10.640721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f300914-bf9d-412d-a734-e5b2b969bf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:10.641105 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f300914-bf9d-412d-a734-e5b2b969bf77 admin admin] PATH_INFO: `/v3/projects/5dda114b023948e7897d63425c5d1065/users/ab5181cda9bb477d8306cd790d0d81c2/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:10.660257 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f300914-bf9d-412d-a734-e5b2b969bf77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dda114b023948e7897d63425c5d1065, user_id=ab5181cda9bb477d8306cd790d0d81c2, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:10.662429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f300914-bf9d-412d-a734-e5b2b969bf77 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:10.785150 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1172/2330] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:52:10 2022] PUT /identity/v3/projects/5dda114b023948e7897d63425c5d1065/users/ab5181cda9bb477d8306cd790d0d81c2/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:52:10.787380 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ec6fd-c019-4ce3-b1cb-da4e7e7b2bea admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:10.801000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ec6fd-c019-4ce3-b1cb-da4e7e7b2bea admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:10.866684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:10.886691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a910ca4-ee29-4134-84ca-83aa66e77003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:10.887542 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a910ca4-ee29-4134-84ca-83aa66e77003 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:10.888107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a910ca4-ee29-4134-84ca-83aa66e77003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:10.888107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a910ca4-ee29-4134-84ca-83aa66e77003 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:10.889098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a910ca4-ee29-4134-84ca-83aa66e77003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:10.891734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a910ca4-ee29-4134-84ca-83aa66e77003 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:10.903100 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1159/2331] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:10.907836 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f300914-bf9d-412d-a734-e5b2b969bf77 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:10.919263 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f300914-bf9d-412d-a734-e5b2b969bf77 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:10.924787 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f6547-004d-4638-aab4-40c087d129dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:10.926209 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f6547-004d-4638-aab4-40c087d129dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:10.928102 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f6547-004d-4638-aab4-40c087d129dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:10.928588 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f6547-004d-4638-aab4-40c087d129dd admin admin] PATH_INFO: `/v3/projects/5dda114b023948e7897d63425c5d1065/users/ab5181cda9bb477d8306cd790d0d81c2/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:10.931281 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f6547-004d-4638-aab4-40c087d129dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dda114b023948e7897d63425c5d1065, user_id=ab5181cda9bb477d8306cd790d0d81c2, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:10.934855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f6547-004d-4638-aab4-40c087d129dd admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:10.961992 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1173/2332] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:52:10 2022] PUT /identity/v3/projects/5dda114b023948e7897d63425c5d1065/users/ab5181cda9bb477d8306cd790d0d81c2/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:52:10.981266 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f672b0e-8790-4f16-9459-39a887befefc None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:10.981561 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f672b0e-8790-4f16-9459-39a887befefc None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:10.981957 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f672b0e-8790-4f16-9459-39a887befefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:11.039500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-7f672b0e-8790-4f16-9459-39a887befefc None None] MFA Rules not processed for user `ab5181cda9bb477d8306cd790d0d81c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:11.040828 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7f672b0e-8790-4f16-9459-39a887befefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:11.126330 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1160/2333] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:10 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:11.138708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f6547-004d-4638-aab4-40c087d129dd admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:11.141536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f82f6547-004d-4638-aab4-40c087d129dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:11.148125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:11.216890 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f6547-004d-4638-aab4-40c087d129dd admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:11.223069 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9fc2e-3ccd-495c-93f0-58d8d18f188d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:11.224095 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9fc2e-3ccd-495c-93f0-58d8d18f188d service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:11.224509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9fc2e-3ccd-495c-93f0-58d8d18f188d service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:11.224910 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9fc2e-3ccd-495c-93f0-58d8d18f188d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:11.240251 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a9fc2e-3ccd-495c-93f0-58d8d18f188d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:11.244056 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a9fc2e-3ccd-495c-93f0-58d8d18f188d service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:11.256508 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1174/2334] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:52:11.488932 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f672b0e-8790-4f16-9459-39a887befefc None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:11.503766 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f672b0e-8790-4f16-9459-39a887befefc None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:11.509543 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfe78f0-5028-435f-8161-1be802cd2bef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:11.510311 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe78f0-5028-435f-8161-1be802cd2bef service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:11.510749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe78f0-5028-435f-8161-1be802cd2bef service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:11.511147 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe78f0-5028-435f-8161-1be802cd2bef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:11.590766 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfe78f0-5028-435f-8161-1be802cd2bef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:11.590766 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfe78f0-5028-435f-8161-1be802cd2bef service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:11.612101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1161/2335] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:52:21.596608 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=70 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:21.616698 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=25 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:24.330884 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9fc2e-3ccd-495c-93f0-58d8d18f188d service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:24.397026 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9fc2e-3ccd-495c-93f0-58d8d18f188d service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:24.401936 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1963ce-1590-4d37-80c0-8b40d5d33824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:24.402849 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1963ce-1590-4d37-80c0-8b40d5d33824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:24.403132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1963ce-1590-4d37-80c0-8b40d5d33824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:24.403409 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1963ce-1590-4d37-80c0-8b40d5d33824 admin admin] PATH_INFO: `/v3/users/f502f724cd4149a89af06a76c2b11db9` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:24.404672 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1963ce-1590-4d37-80c0-8b40d5d33824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f502f724cd4149a89af06a76c2b11db9)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:24.406635 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1963ce-1590-4d37-80c0-8b40d5d33824 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:24.576240 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7b1963ce-1590-4d37-80c0-8b40d5d33824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f502f724cd4149a89af06a76c2b11db9'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:24.583235 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7b1963ce-1590-4d37-80c0-8b40d5d33824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f502f724cd4149a89af06a76c2b11db9'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:24.583939 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7b1963ce-1590-4d37-80c0-8b40d5d33824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f502f724cd4149a89af06a76c2b11db9'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:24.584549 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7b1963ce-1590-4d37-80c0-8b40d5d33824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f502f724cd4149a89af06a76c2b11db9'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:24.596903 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7b1963ce-1590-4d37-80c0-8b40d5d33824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f502f724cd4149a89af06a76c2b11db9'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:24.665449 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1175/2336] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:52:24 2022] DELETE /identity/v3/users/f502f724cd4149a89af06a76c2b11db9 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:24.669955 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5edce-989c-49d0-8483-ac9c82aca83a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:24.670470 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5edce-989c-49d0-8483-ac9c82aca83a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:24.670602 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5edce-989c-49d0-8483-ac9c82aca83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:24.687924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-dfa5edce-989c-49d0-8483-ac9c82aca83a None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:24.689091 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-dfa5edce-989c-49d0-8483-ac9c82aca83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:24.748538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1162/2337] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:24.758803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1963ce-1590-4d37-80c0-8b40d5d33824 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:24.760208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-7b1963ce-1590-4d37-80c0-8b40d5d33824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:24.817813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1963ce-1590-4d37-80c0-8b40d5d33824 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:24.821419 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e34f1-2862-448f-9fc6-d7e7888cef60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:24.822054 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e34f1-2862-448f-9fc6-d7e7888cef60 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:24.822337 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e34f1-2862-448f-9fc6-d7e7888cef60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:24.822645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e34f1-2862-448f-9fc6-d7e7888cef60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:24.831531 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e34f1-2862-448f-9fc6-d7e7888cef60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:24.832905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e34f1-2862-448f-9fc6-d7e7888cef60 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:24.842060 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1176/2338] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:52:25.310854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5edce-989c-49d0-8483-ac9c82aca83a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:25.312603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-dfa5edce-989c-49d0-8483-ac9c82aca83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:25.328589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5edce-989c-49d0-8483-ac9c82aca83a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:25.340492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51afc9-f022-469e-8963-26a9d7f445d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:25.341959 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51afc9-f022-469e-8963-26a9d7f445d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:25.342154 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51afc9-f022-469e-8963-26a9d7f445d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:25.342403 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51afc9-f022-469e-8963-26a9d7f445d1 admin admin] PATH_INFO: `/v3/projects/a7ac39b34db144abbca29cf69555e86b` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:25.343689 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51afc9-f022-469e-8963-26a9d7f445d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7ac39b34db144abbca29cf69555e86b)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:25.346129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51afc9-f022-469e-8963-26a9d7f445d1 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:25.506978 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3e51afc9-f022-469e-8963-26a9d7f445d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7ac39b34db144abbca29cf69555e86b', 'request_id': 'req-3e51afc9-f022-469e-8963-26a9d7f445d1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:25.513181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3e51afc9-f022-469e-8963-26a9d7f445d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7ac39b34db144abbca29cf69555e86b', 'request_id': 'req-3e51afc9-f022-469e-8963-26a9d7f445d1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:25.597030 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3e51afc9-f022-469e-8963-26a9d7f445d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7ac39b34db144abbca29cf69555e86b', 'request_id': 'req-3e51afc9-f022-469e-8963-26a9d7f445d1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:25.608766 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3e51afc9-f022-469e-8963-26a9d7f445d1 admin admin] The token cache is being invalidate because project a7ac39b34db144abbca29cf69555e86b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:52:25.608988 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3e51afc9-f022-469e-8963-26a9d7f445d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:25.609637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3e51afc9-f022-469e-8963-26a9d7f445d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:25.611256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1163/2339] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:52:25 2022] DELETE /identity/v3/projects/a7ac39b34db144abbca29cf69555e86b => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:25.615483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e34f1-2862-448f-9fc6-d7e7888cef60 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:25.625524 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e34f1-2862-448f-9fc6-d7e7888cef60 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:25.646228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:25.685017 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc92d1-a91e-485a-b47b-e7ab0ead488e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:25.685897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc92d1-a91e-485a-b47b-e7ab0ead488e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:25.686180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc92d1-a91e-485a-b47b-e7ab0ead488e admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:25.686456 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc92d1-a91e-485a-b47b-e7ab0ead488e admin admin] PATH_INFO: `/v3/users/ab5181cda9bb477d8306cd790d0d81c2` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:25.687742 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc92d1-a91e-485a-b47b-e7ab0ead488e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab5181cda9bb477d8306cd790d0d81c2)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:25.690290 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc92d1-a91e-485a-b47b-e7ab0ead488e admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:25.913526 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-90dc92d1-a91e-485a-b47b-e7ab0ead488e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab5181cda9bb477d8306cd790d0d81c2'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:25.920773 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-90dc92d1-a91e-485a-b47b-e7ab0ead488e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab5181cda9bb477d8306cd790d0d81c2'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:25.921445 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-90dc92d1-a91e-485a-b47b-e7ab0ead488e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab5181cda9bb477d8306cd790d0d81c2'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:25.922062 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-90dc92d1-a91e-485a-b47b-e7ab0ead488e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab5181cda9bb477d8306cd790d0d81c2'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:25.934970 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-90dc92d1-a91e-485a-b47b-e7ab0ead488e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab5181cda9bb477d8306cd790d0d81c2'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:25.994630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1177/2340] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:52:25 2022] DELETE /identity/v3/users/ab5181cda9bb477d8306cd790d0d81c2 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:26.525260 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51afc9-f022-469e-8963-26a9d7f445d1 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:26.526883 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3e51afc9-f022-469e-8963-26a9d7f445d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:26.541709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51afc9-f022-469e-8963-26a9d7f445d1 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:26.602921 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b994d-a185-4c85-a951-d3687572f971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:26.603936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b994d-a185-4c85-a951-d3687572f971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:26.604225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b994d-a185-4c85-a951-d3687572f971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:26.604529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b994d-a185-4c85-a951-d3687572f971 admin admin] PATH_INFO: `/v3/projects/5dda114b023948e7897d63425c5d1065` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:26.605742 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b994d-a185-4c85-a951-d3687572f971 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dda114b023948e7897d63425c5d1065)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:26.607741 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b994d-a185-4c85-a951-d3687572f971 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:26.719245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:26.741470 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-387b994d-a185-4c85-a951-d3687572f971 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dda114b023948e7897d63425c5d1065', 'request_id': 'req-387b994d-a185-4c85-a951-d3687572f971'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:26.748917 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-387b994d-a185-4c85-a951-d3687572f971 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dda114b023948e7897d63425c5d1065', 'request_id': 'req-387b994d-a185-4c85-a951-d3687572f971'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:26.820458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-387b994d-a185-4c85-a951-d3687572f971 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dda114b023948e7897d63425c5d1065', 'request_id': 'req-387b994d-a185-4c85-a951-d3687572f971'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:26.833775 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-387b994d-a185-4c85-a951-d3687572f971 admin admin] The token cache is being invalidate because project 5dda114b023948e7897d63425c5d1065 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:52:26.834070 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-387b994d-a185-4c85-a951-d3687572f971 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:26.834775 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-387b994d-a185-4c85-a951-d3687572f971 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:26.836351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1164/2341] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:52:26 2022] DELETE /identity/v3/projects/5dda114b023948e7897d63425c5d1065 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:26.841395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df96f7-9703-4e11-8449-f6bb64105e9d None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:26.841883 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df96f7-9703-4e11-8449-f6bb64105e9d None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:26.841979 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df96f7-9703-4e11-8449-f6bb64105e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:26.860707 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-54df96f7-9703-4e11-8449-f6bb64105e9d None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:26.861964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-54df96f7-9703-4e11-8449-f6bb64105e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:26.924939 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1178/2342] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:26.932902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a474e-3094-41ac-bf91-353c9d7db6e2 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:26.933199 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a474e-3094-41ac-bf91-353c9d7db6e2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:26.933482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a474e-3094-41ac-bf91-353c9d7db6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:26.952866 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-963a474e-3094-41ac-bf91-353c9d7db6e2 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:26.954247 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-963a474e-3094-41ac-bf91-353c9d7db6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:26.962994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1165/2343] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:26.967228 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df96f7-9703-4e11-8449-f6bb64105e9d None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:26.977211 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df96f7-9703-4e11-8449-f6bb64105e9d None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:26.981889 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb58dbc-991d-4d8c-bc69-7d3e021f6c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:26.982567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb58dbc-991d-4d8c-bc69-7d3e021f6c23 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:26.982864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb58dbc-991d-4d8c-bc69-7d3e021f6c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:26.983144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb58dbc-991d-4d8c-bc69-7d3e021f6c23 admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:26.983869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb58dbc-991d-4d8c-bc69-7d3e021f6c23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:52:26.984178 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb58dbc-991d-4d8c-bc69-7d3e021f6c23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:26.985766 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb58dbc-991d-4d8c-bc69-7d3e021f6c23 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:27.000440 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1179/2344] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:52:27.002927 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a474e-3094-41ac-bf91-353c9d7db6e2 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:27.012259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a474e-3094-41ac-bf91-353c9d7db6e2 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:27.017071 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f765db3-ac84-4497-b51d-dc9747bf418a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:27.017802 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f765db3-ac84-4497-b51d-dc9747bf418a admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:27.018628 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f765db3-ac84-4497-b51d-dc9747bf418a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:27.018904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f765db3-ac84-4497-b51d-dc9747bf418a admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:27.019562 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f765db3-ac84-4497-b51d-dc9747bf418a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:27.021483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f765db3-ac84-4497-b51d-dc9747bf418a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:27.102476 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1166/2345] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:52:27 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:52:27.106633 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb58dbc-991d-4d8c-bc69-7d3e021f6c23 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:27.118000 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb58dbc-991d-4d8c-bc69-7d3e021f6c23 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:27.145746 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:27.178365 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd1b63-a8df-4617-b68a-7268af59a17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:27.179219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd1b63-a8df-4617-b68a-7268af59a17d admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:27.179504 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd1b63-a8df-4617-b68a-7268af59a17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:27.179796 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd1b63-a8df-4617-b68a-7268af59a17d admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:27.180488 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd1b63-a8df-4617-b68a-7268af59a17d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:27.182407 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd1b63-a8df-4617-b68a-7268af59a17d admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:27.277395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1180/2346] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:52:27 2022] POST /identity/v3/users => generated 283 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:52:27.281414 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f765db3-ac84-4497-b51d-dc9747bf418a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:27.291887 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f765db3-ac84-4497-b51d-dc9747bf418a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:27.296665 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b38f9b-8586-4df9-8295-7f41793bca1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:27.297401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b38f9b-8586-4df9-8295-7f41793bca1a admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:27.297779 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b38f9b-8586-4df9-8295-7f41793bca1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:27.298128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b38f9b-8586-4df9-8295-7f41793bca1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:27.298965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b38f9b-8586-4df9-8295-7f41793bca1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:27.301165 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b38f9b-8586-4df9-8295-7f41793bca1a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:27.312735 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1167/2347] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:27.317414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd1b63-a8df-4617-b68a-7268af59a17d admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:27.327688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd1b63-a8df-4617-b68a-7268af59a17d admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:27.332672 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ea540-00e0-44d2-b226-5faa79ad78c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:27.333472 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ea540-00e0-44d2-b226-5faa79ad78c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:27.333763 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ea540-00e0-44d2-b226-5faa79ad78c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:27.334040 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ea540-00e0-44d2-b226-5faa79ad78c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:27.334802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ea540-00e0-44d2-b226-5faa79ad78c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:27.336428 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ea540-00e0-44d2-b226-5faa79ad78c3 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:27.348459 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1181/2348] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:27.353958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b38f9b-8586-4df9-8295-7f41793bca1a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:27.365097 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b38f9b-8586-4df9-8295-7f41793bca1a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:27.369992 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0d71d-8c8c-4e99-b17b-8565f04441f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:27.370924 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0d71d-8c8c-4e99-b17b-8565f04441f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:27.371204 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0d71d-8c8c-4e99-b17b-8565f04441f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:27.371496 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0d71d-8c8c-4e99-b17b-8565f04441f4 admin admin] PATH_INFO: `/v3/projects/200150b7ee454899a66276e7aa72d3c3/users/afed0778ad1c4153885331b5136d4d58/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:27.387038 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0d71d-8c8c-4e99-b17b-8565f04441f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=200150b7ee454899a66276e7aa72d3c3, user_id=afed0778ad1c4153885331b5136d4d58, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:27.389802 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0d71d-8c8c-4e99-b17b-8565f04441f4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:27.459238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1168/2349] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:52:27 2022] PUT /identity/v3/projects/200150b7ee454899a66276e7aa72d3c3/users/afed0778ad1c4153885331b5136d4d58/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:52:27.470229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18e78d-eaff-4845-af71-032c48ad7591 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:27.470518 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18e78d-eaff-4845-af71-032c48ad7591 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:27.470838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18e78d-eaff-4845-af71-032c48ad7591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:27.498476 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-0b18e78d-eaff-4845-af71-032c48ad7591 None None] MFA Rules not processed for user `afed0778ad1c4153885331b5136d4d58`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:27.499697 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-0b18e78d-eaff-4845-af71-032c48ad7591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:27.567843 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1182/2350] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:27.572154 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0d71d-8c8c-4e99-b17b-8565f04441f4 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:27.634801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0d71d-8c8c-4e99-b17b-8565f04441f4 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:27.639485 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b528f01-c05b-4f11-b73c-8d427a2a0b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:27.640148 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b528f01-c05b-4f11-b73c-8d427a2a0b05 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:27.640435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b528f01-c05b-4f11-b73c-8d427a2a0b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:27.640716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b528f01-c05b-4f11-b73c-8d427a2a0b05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:27.641400 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b528f01-c05b-4f11-b73c-8d427a2a0b05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:27.643597 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b528f01-c05b-4f11-b73c-8d427a2a0b05 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:27.773108 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1169/2351] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:52:27 2022] POST /identity/v3/projects => generated 365 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:52:27.777192 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18e78d-eaff-4845-af71-032c48ad7591 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:27.788674 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18e78d-eaff-4845-af71-032c48ad7591 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:27.843022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b612a-821c-465e-a6c6-30f6afab7150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:27.843682 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b612a-821c-465e-a6c6-30f6afab7150 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:27.843962 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b612a-821c-465e-a6c6-30f6afab7150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:27.844235 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b612a-821c-465e-a6c6-30f6afab7150 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:27.844919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b612a-821c-465e-a6c6-30f6afab7150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:27.847384 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b612a-821c-465e-a6c6-30f6afab7150 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:27.933777 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1183/2352] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:52:27 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:52:27.937771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b528f01-c05b-4f11-b73c-8d427a2a0b05 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:27.948187 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b528f01-c05b-4f11-b73c-8d427a2a0b05 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:27.952810 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41484e-82bc-4576-9740-f8678c16f8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:27.953469 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41484e-82bc-4576-9740-f8678c16f8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:27.953884 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41484e-82bc-4576-9740-f8678c16f8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:27.954173 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41484e-82bc-4576-9740-f8678c16f8f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:27.955017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41484e-82bc-4576-9740-f8678c16f8f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:27.957364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41484e-82bc-4576-9740-f8678c16f8f0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:27.968388 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1170/2353] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:27.972965 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b612a-821c-465e-a6c6-30f6afab7150 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:27.983077 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b612a-821c-465e-a6c6-30f6afab7150 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:27.988625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454719f0-8a26-4690-bed0-3c4c5d2abfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:27.989505 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454719f0-8a26-4690-bed0-3c4c5d2abfd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:27.989783 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454719f0-8a26-4690-bed0-3c4c5d2abfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:27.990140 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454719f0-8a26-4690-bed0-3c4c5d2abfd7 admin admin] PATH_INFO: `/v3/projects/144a667a546c4ad99807a8d016cb79ba/users/00d02fc6131a4538880e883692d86c87/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:28.005023 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454719f0-8a26-4690-bed0-3c4c5d2abfd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=144a667a546c4ad99807a8d016cb79ba, user_id=00d02fc6131a4538880e883692d86c87, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:28.007739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454719f0-8a26-4690-bed0-3c4c5d2abfd7 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:28.086165 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1184/2354] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:52:27 2022] PUT /identity/v3/projects/144a667a546c4ad99807a8d016cb79ba/users/00d02fc6131a4538880e883692d86c87/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:52:28.090091 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41484e-82bc-4576-9740-f8678c16f8f0 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:28.100959 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41484e-82bc-4576-9740-f8678c16f8f0 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:28.121460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:28.158627 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f7ce4-ac35-47e6-bff1-6cec4a38c05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:28.160232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f7ce4-ac35-47e6-bff1-6cec4a38c05e admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:28.160515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f7ce4-ac35-47e6-bff1-6cec4a38c05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:28.160786 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f7ce4-ac35-47e6-bff1-6cec4a38c05e admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:28.161502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f7ce4-ac35-47e6-bff1-6cec4a38c05e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:28.163139 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f7ce4-ac35-47e6-bff1-6cec4a38c05e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:28.173063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1171/2355] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:28.177321 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454719f0-8a26-4690-bed0-3c4c5d2abfd7 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:28.182914 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:28.188198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454719f0-8a26-4690-bed0-3c4c5d2abfd7 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:28.192999 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed5879-5238-4e6d-baf8-1db9d101e5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:28.193882 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed5879-5238-4e6d-baf8-1db9d101e5d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:28.194225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed5879-5238-4e6d-baf8-1db9d101e5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:28.194497 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed5879-5238-4e6d-baf8-1db9d101e5d5 admin admin] PATH_INFO: `/v3/projects/144a667a546c4ad99807a8d016cb79ba/users/00d02fc6131a4538880e883692d86c87/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:28.196489 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed5879-5238-4e6d-baf8-1db9d101e5d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=144a667a546c4ad99807a8d016cb79ba, user_id=00d02fc6131a4538880e883692d86c87, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:28.199256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed5879-5238-4e6d-baf8-1db9d101e5d5 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:28.210237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1185/2356] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:52:28 2022] PUT /identity/v3/projects/144a667a546c4ad99807a8d016cb79ba/users/00d02fc6131a4538880e883692d86c87/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:28.222029 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f008a-6e3e-4d27-8b3d-2aed9af6f904 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:28.222349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f008a-6e3e-4d27-8b3d-2aed9af6f904 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:28.222670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f008a-6e3e-4d27-8b3d-2aed9af6f904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:28.248664 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-0f3f008a-6e3e-4d27-8b3d-2aed9af6f904 None None] MFA Rules not processed for user `00d02fc6131a4538880e883692d86c87`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:28.249857 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-0f3f008a-6e3e-4d27-8b3d-2aed9af6f904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:28.339038 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1172/2357] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:28 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:52:28.351589 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed5879-5238-4e6d-baf8-1db9d101e5d5 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:28.353217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e6ed5879-5238-4e6d-baf8-1db9d101e5d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:28.411232 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed5879-5238-4e6d-baf8-1db9d101e5d5 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:28.414953 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd19afb-0086-453a-9b2e-ff8117d72f6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:28.415589 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd19afb-0086-453a-9b2e-ff8117d72f6d service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:28.415869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd19afb-0086-453a-9b2e-ff8117d72f6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:28.416142 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd19afb-0086-453a-9b2e-ff8117d72f6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:28.425170 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd19afb-0086-453a-9b2e-ff8117d72f6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:28.426679 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd19afb-0086-453a-9b2e-ff8117d72f6d service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:28.435822 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1186/2358] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:28 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:52:29.310669 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f008a-6e3e-4d27-8b3d-2aed9af6f904 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:29.373921 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f008a-6e3e-4d27-8b3d-2aed9af6f904 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:29.379210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbe6a7-5d8e-427d-b3f9-f8c72accca77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:29.380074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbe6a7-5d8e-427d-b3f9-f8c72accca77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:29.380361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbe6a7-5d8e-427d-b3f9-f8c72accca77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:29.380636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbe6a7-5d8e-427d-b3f9-f8c72accca77 admin admin] PATH_INFO: `/v3/users/afed0778ad1c4153885331b5136d4d58` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:29.381810 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fbe6a7-5d8e-427d-b3f9-f8c72accca77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afed0778ad1c4153885331b5136d4d58)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:29.383771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fbe6a7-5d8e-427d-b3f9-f8c72accca77 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:29.555150 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-23fbe6a7-5d8e-427d-b3f9-f8c72accca77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afed0778ad1c4153885331b5136d4d58'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:29.556329 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:29.562471 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-23fbe6a7-5d8e-427d-b3f9-f8c72accca77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afed0778ad1c4153885331b5136d4d58'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:29.563293 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-23fbe6a7-5d8e-427d-b3f9-f8c72accca77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afed0778ad1c4153885331b5136d4d58'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:29.563901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-23fbe6a7-5d8e-427d-b3f9-f8c72accca77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afed0778ad1c4153885331b5136d4d58'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:29.576018 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-23fbe6a7-5d8e-427d-b3f9-f8c72accca77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afed0778ad1c4153885331b5136d4d58'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:29.654558 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1173/2359] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:52:29 2022] DELETE /identity/v3/users/afed0778ad1c4153885331b5136d4d58 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:29.659763 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28852dff-9631-45ef-a85f-d85cb1ef7843 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:29.660359 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28852dff-9631-45ef-a85f-d85cb1ef7843 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:29.660359 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28852dff-9631-45ef-a85f-d85cb1ef7843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:29.679597 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-28852dff-9631-45ef-a85f-d85cb1ef7843 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:29.680802 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-28852dff-9631-45ef-a85f-d85cb1ef7843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:29.739002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1187/2360] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:29.748812 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbe6a7-5d8e-427d-b3f9-f8c72accca77 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:29.750233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-23fbe6a7-5d8e-427d-b3f9-f8c72accca77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:29.807853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbe6a7-5d8e-427d-b3f9-f8c72accca77 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:29.811531 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac7842-4da9-43de-8c34-39bbebe515ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:29.812166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac7842-4da9-43de-8c34-39bbebe515ab service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:29.812448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac7842-4da9-43de-8c34-39bbebe515ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:29.812725 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac7842-4da9-43de-8c34-39bbebe515ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:29.821856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac7842-4da9-43de-8c34-39bbebe515ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:29.823310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac7842-4da9-43de-8c34-39bbebe515ab service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:29.833377 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1174/2361] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:52:30.249505 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28852dff-9631-45ef-a85f-d85cb1ef7843 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:30.250982 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-28852dff-9631-45ef-a85f-d85cb1ef7843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:30.264781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28852dff-9631-45ef-a85f-d85cb1ef7843 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:30.270302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c326418-cff0-42c8-a831-f964416cd0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:30.271238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c326418-cff0-42c8-a831-f964416cd0e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:30.271509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c326418-cff0-42c8-a831-f964416cd0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:30.271776 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c326418-cff0-42c8-a831-f964416cd0e0 admin admin] PATH_INFO: `/v3/projects/200150b7ee454899a66276e7aa72d3c3` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:30.272952 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c326418-cff0-42c8-a831-f964416cd0e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=200150b7ee454899a66276e7aa72d3c3)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:30.274916 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c326418-cff0-42c8-a831-f964416cd0e0 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:30.368044 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-0c326418-cff0-42c8-a831-f964416cd0e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '200150b7ee454899a66276e7aa72d3c3', 'request_id': 'req-0c326418-cff0-42c8-a831-f964416cd0e0'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:30.373941 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-0c326418-cff0-42c8-a831-f964416cd0e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '200150b7ee454899a66276e7aa72d3c3', 'request_id': 'req-0c326418-cff0-42c8-a831-f964416cd0e0'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:30.413652 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-0c326418-cff0-42c8-a831-f964416cd0e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '200150b7ee454899a66276e7aa72d3c3', 'request_id': 'req-0c326418-cff0-42c8-a831-f964416cd0e0'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:30.423944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-0c326418-cff0-42c8-a831-f964416cd0e0 admin admin] The token cache is being invalidate because project 200150b7ee454899a66276e7aa72d3c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:52:30.424227 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-0c326418-cff0-42c8-a831-f964416cd0e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:30.424875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-0c326418-cff0-42c8-a831-f964416cd0e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:30.426434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1188/2362] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:52:30 2022] DELETE /identity/v3/projects/200150b7ee454899a66276e7aa72d3c3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:30.430394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac7842-4da9-43de-8c34-39bbebe515ab service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:30.440738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac7842-4da9-43de-8c34-39bbebe515ab service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:30.498265 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3715be-de76-4833-acd5-84665c0612af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:30.499173 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3715be-de76-4833-acd5-84665c0612af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:30.499456 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3715be-de76-4833-acd5-84665c0612af admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:30.499726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3715be-de76-4833-acd5-84665c0612af admin admin] PATH_INFO: `/v3/users/00d02fc6131a4538880e883692d86c87` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:30.500883 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3715be-de76-4833-acd5-84665c0612af admin admin] RBAC: Authorizing `identity:delete_user(user_id=00d02fc6131a4538880e883692d86c87)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:30.503481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3715be-de76-4833-acd5-84665c0612af admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:30.665021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ea3715be-de76-4833-acd5-84665c0612af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00d02fc6131a4538880e883692d86c87'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:30.674180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ea3715be-de76-4833-acd5-84665c0612af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00d02fc6131a4538880e883692d86c87'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:30.675529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ea3715be-de76-4833-acd5-84665c0612af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00d02fc6131a4538880e883692d86c87'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:30.676210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ea3715be-de76-4833-acd5-84665c0612af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00d02fc6131a4538880e883692d86c87'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:30.686516 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:30.688793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ea3715be-de76-4833-acd5-84665c0612af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00d02fc6131a4538880e883692d86c87'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:30.790837 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1175/2363] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:52:30 2022] DELETE /identity/v3/users/00d02fc6131a4538880e883692d86c87 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:31.176190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c326418-cff0-42c8-a831-f964416cd0e0 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:31.177676 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-0c326418-cff0-42c8-a831-f964416cd0e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:31.193593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c326418-cff0-42c8-a831-f964416cd0e0 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:31.206912 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:31.252020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce358f2-49a9-4fc7-8c54-a9873c14b6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:31.253006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce358f2-49a9-4fc7-8c54-a9873c14b6d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:31.253294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce358f2-49a9-4fc7-8c54-a9873c14b6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:31.253564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce358f2-49a9-4fc7-8c54-a9873c14b6d1 admin admin] PATH_INFO: `/v3/projects/144a667a546c4ad99807a8d016cb79ba` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:31.254793 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce358f2-49a9-4fc7-8c54-a9873c14b6d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=144a667a546c4ad99807a8d016cb79ba)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:31.256772 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce358f2-49a9-4fc7-8c54-a9873c14b6d1 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:31.434943 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-dce358f2-49a9-4fc7-8c54-a9873c14b6d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '144a667a546c4ad99807a8d016cb79ba', 'request_id': 'req-dce358f2-49a9-4fc7-8c54-a9873c14b6d1'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:31.441092 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-dce358f2-49a9-4fc7-8c54-a9873c14b6d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '144a667a546c4ad99807a8d016cb79ba', 'request_id': 'req-dce358f2-49a9-4fc7-8c54-a9873c14b6d1'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:31.514636 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-dce358f2-49a9-4fc7-8c54-a9873c14b6d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '144a667a546c4ad99807a8d016cb79ba', 'request_id': 'req-dce358f2-49a9-4fc7-8c54-a9873c14b6d1'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:31.526066 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-dce358f2-49a9-4fc7-8c54-a9873c14b6d1 admin admin] The token cache is being invalidate because project 144a667a546c4ad99807a8d016cb79ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:52:31.526354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-dce358f2-49a9-4fc7-8c54-a9873c14b6d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:31.527015 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-dce358f2-49a9-4fc7-8c54-a9873c14b6d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:31.528549 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1189/2364] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:52:31 2022] DELETE /identity/v3/projects/144a667a546c4ad99807a8d016cb79ba => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:31.533580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a521f5-e616-4b1c-bf11-ec286311b2db None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:31.535084 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a521f5-e616-4b1c-bf11-ec286311b2db None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:31.535084 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a521f5-e616-4b1c-bf11-ec286311b2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:31.552646 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-00a521f5-e616-4b1c-bf11-ec286311b2db None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:31.553918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-00a521f5-e616-4b1c-bf11-ec286311b2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:31.613118 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1176/2365] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:31.620892 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bd5f1-aacb-4641-abb8-e4e1f88c91a2 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:31.622390 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bd5f1-aacb-4641-abb8-e4e1f88c91a2 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:31.622390 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bd5f1-aacb-4641-abb8-e4e1f88c91a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:31.653626 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-e26bd5f1-aacb-4641-abb8-e4e1f88c91a2 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:31.654954 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e26bd5f1-aacb-4641-abb8-e4e1f88c91a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:31.664534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1190/2366] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:31.668935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a521f5-e616-4b1c-bf11-ec286311b2db None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:31.680354 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a521f5-e616-4b1c-bf11-ec286311b2db None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:31.685508 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce62bb7-a920-44e1-9d32-21c76cd49df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:31.691197 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce62bb7-a920-44e1-9d32-21c76cd49df9 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:31.691197 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce62bb7-a920-44e1-9d32-21c76cd49df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:31.691197 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce62bb7-a920-44e1-9d32-21c76cd49df9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:31.691197 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce62bb7-a920-44e1-9d32-21c76cd49df9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:52:31.691197 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce62bb7-a920-44e1-9d32-21c76cd49df9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:31.691197 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce62bb7-a920-44e1-9d32-21c76cd49df9 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:31.702169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1177/2367] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:52:31.707420 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bd5f1-aacb-4641-abb8-e4e1f88c91a2 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:31.716876 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bd5f1-aacb-4641-abb8-e4e1f88c91a2 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:31.722069 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd108b28-e98e-4450-8398-0f6b34179620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:31.725669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd108b28-e98e-4450-8398-0f6b34179620 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:31.725669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd108b28-e98e-4450-8398-0f6b34179620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:31.725669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd108b28-e98e-4450-8398-0f6b34179620 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:31.725669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd108b28-e98e-4450-8398-0f6b34179620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:31.726172 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd108b28-e98e-4450-8398-0f6b34179620 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:31.811004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1191/2368] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:52:31 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:52:31.812387 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce62bb7-a920-44e1-9d32-21c76cd49df9 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:31.823361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce62bb7-a920-44e1-9d32-21c76cd49df9 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:31.881733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad6f8f-d84d-415a-939b-325f32b9f7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:31.882401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad6f8f-d84d-415a-939b-325f32b9f7b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:31.882717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad6f8f-d84d-415a-939b-325f32b9f7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:31.883002 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad6f8f-d84d-415a-939b-325f32b9f7b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:31.883688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad6f8f-d84d-415a-939b-325f32b9f7b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:31.885610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad6f8f-d84d-415a-939b-325f32b9f7b1 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:31.968072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1178/2369] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:52:31 2022] POST /identity/v3/users => generated 285 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:52:31.972729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd108b28-e98e-4450-8398-0f6b34179620 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:31.983206 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd108b28-e98e-4450-8398-0f6b34179620 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:31.988075 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eb1e9-1a97-4124-ab30-675b26e0eec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:31.988722 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb1e9-1a97-4124-ab30-675b26e0eec9 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:31.989001 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb1e9-1a97-4124-ab30-675b26e0eec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:31.989277 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb1e9-1a97-4124-ab30-675b26e0eec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:31.989992 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9eb1e9-1a97-4124-ab30-675b26e0eec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:31.991610 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9eb1e9-1a97-4124-ab30-675b26e0eec9 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:32.001757 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1192/2370] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:32.006157 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad6f8f-d84d-415a-939b-325f32b9f7b1 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:32.016127 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad6f8f-d84d-415a-939b-325f32b9f7b1 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:32.021604 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c0b4b-1d65-4c40-8dcd-b4c29d1efe1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:32.022252 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c0b4b-1d65-4c40-8dcd-b4c29d1efe1a admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:32.022555 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c0b4b-1d65-4c40-8dcd-b4c29d1efe1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:32.022837 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c0b4b-1d65-4c40-8dcd-b4c29d1efe1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:32.023586 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c0b4b-1d65-4c40-8dcd-b4c29d1efe1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:32.025207 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c0b4b-1d65-4c40-8dcd-b4c29d1efe1a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:32.035352 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1179/2371] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:32.039288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eb1e9-1a97-4124-ab30-675b26e0eec9 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:32.049158 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eb1e9-1a97-4124-ab30-675b26e0eec9 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:32.053907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434896e0-269a-4076-a8f1-f5de166996fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:32.054816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434896e0-269a-4076-a8f1-f5de166996fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:32.055101 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434896e0-269a-4076-a8f1-f5de166996fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:32.055379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434896e0-269a-4076-a8f1-f5de166996fb admin admin] PATH_INFO: `/v3/projects/a3307c176a7747878d7e48600d69522b/users/ae1923f6c730415aa0e3be9e0fa07dac/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:32.070174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434896e0-269a-4076-a8f1-f5de166996fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3307c176a7747878d7e48600d69522b, user_id=ae1923f6c730415aa0e3be9e0fa07dac, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:32.072920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434896e0-269a-4076-a8f1-f5de166996fb admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:32.167947 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1193/2372] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:52:32 2022] PUT /identity/v3/projects/a3307c176a7747878d7e48600d69522b/users/ae1923f6c730415aa0e3be9e0fa07dac/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:32.179609 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e246ce2-3cd2-436e-8a46-cbe612add350 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:32.179903 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e246ce2-3cd2-436e-8a46-cbe612add350 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:32.180187 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e246ce2-3cd2-436e-8a46-cbe612add350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:32.207265 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-4e246ce2-3cd2-436e-8a46-cbe612add350 None None] MFA Rules not processed for user `ae1923f6c730415aa0e3be9e0fa07dac`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:32.208500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4e246ce2-3cd2-436e-8a46-cbe612add350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:32.249910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:32.272591 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1180/2373] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:32 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:32.277104 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434896e0-269a-4076-a8f1-f5de166996fb admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:32.322057 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:32.339213 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434896e0-269a-4076-a8f1-f5de166996fb admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:32.344340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e80c9-f639-4d02-9066-d212e4459469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:32.345046 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e80c9-f639-4d02-9066-d212e4459469 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:32.345046 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e80c9-f639-4d02-9066-d212e4459469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:32.345247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e80c9-f639-4d02-9066-d212e4459469 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:32.345925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e80c9-f639-4d02-9066-d212e4459469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:32.347933 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e80c9-f639-4d02-9066-d212e4459469 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:32.478451 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1194/2374] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:52:32 2022] POST /identity/v3/projects => generated 367 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:52:32.482511 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e246ce2-3cd2-436e-8a46-cbe612add350 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:32.494334 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e246ce2-3cd2-436e-8a46-cbe612add350 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:32.549082 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ff97e-0b48-4cad-ba0a-f4e6f8405812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:32.550331 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ff97e-0b48-4cad-ba0a-f4e6f8405812 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:32.550650 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ff97e-0b48-4cad-ba0a-f4e6f8405812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:32.550927 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ff97e-0b48-4cad-ba0a-f4e6f8405812 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:32.551624 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ff97e-0b48-4cad-ba0a-f4e6f8405812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:32.553545 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ff97e-0b48-4cad-ba0a-f4e6f8405812 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:32.635076 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1181/2375] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:52:32 2022] POST /identity/v3/users => generated 290 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:52:32.639260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e80c9-f639-4d02-9066-d212e4459469 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:32.649613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e80c9-f639-4d02-9066-d212e4459469 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:32.654211 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259a5f9-3997-4882-a6f0-38897aa22d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:32.654918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259a5f9-3997-4882-a6f0-38897aa22d8e admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:32.655206 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259a5f9-3997-4882-a6f0-38897aa22d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:32.655480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259a5f9-3997-4882-a6f0-38897aa22d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:32.656273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9259a5f9-3997-4882-a6f0-38897aa22d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:32.657865 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9259a5f9-3997-4882-a6f0-38897aa22d8e admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:32.669218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1195/2376] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:32.673344 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ff97e-0b48-4cad-ba0a-f4e6f8405812 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:32.684069 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ff97e-0b48-4cad-ba0a-f4e6f8405812 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:32.688624 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2156df2-6d02-4e10-93aa-92d351426b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:32.689494 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2156df2-6d02-4e10-93aa-92d351426b62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:32.689774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2156df2-6d02-4e10-93aa-92d351426b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:32.690127 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2156df2-6d02-4e10-93aa-92d351426b62 admin admin] PATH_INFO: `/v3/projects/40cec83c2c47499bb24fe889373dafe3/users/19d02716c4dd49c7b292016a118002d1/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:32.704961 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2156df2-6d02-4e10-93aa-92d351426b62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40cec83c2c47499bb24fe889373dafe3, user_id=19d02716c4dd49c7b292016a118002d1, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:32.707914 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2156df2-6d02-4e10-93aa-92d351426b62 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:32.785877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1182/2377] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:52:32 2022] PUT /identity/v3/projects/40cec83c2c47499bb24fe889373dafe3/users/19d02716c4dd49c7b292016a118002d1/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:32.789771 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259a5f9-3997-4882-a6f0-38897aa22d8e admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:32.800442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259a5f9-3997-4882-a6f0-38897aa22d8e admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:32.857595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add09a02-0a99-4d4c-93e2-5f32466c5e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:32.858252 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add09a02-0a99-4d4c-93e2-5f32466c5e70 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:32.858561 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add09a02-0a99-4d4c-93e2-5f32466c5e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:32.858848 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add09a02-0a99-4d4c-93e2-5f32466c5e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:32.859592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add09a02-0a99-4d4c-93e2-5f32466c5e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:32.861193 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add09a02-0a99-4d4c-93e2-5f32466c5e70 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:32.871031 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1196/2378] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:32.875096 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2156df2-6d02-4e10-93aa-92d351426b62 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:32.885355 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2156df2-6d02-4e10-93aa-92d351426b62 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:32.889850 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7d8dd-579f-4ed6-a5e5-1a3ca2d42047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:32.890760 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7d8dd-579f-4ed6-a5e5-1a3ca2d42047 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:32.891044 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7d8dd-579f-4ed6-a5e5-1a3ca2d42047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:32.891319 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7d8dd-579f-4ed6-a5e5-1a3ca2d42047 admin admin] PATH_INFO: `/v3/projects/40cec83c2c47499bb24fe889373dafe3/users/19d02716c4dd49c7b292016a118002d1/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:32.893866 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f7d8dd-579f-4ed6-a5e5-1a3ca2d42047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40cec83c2c47499bb24fe889373dafe3, user_id=19d02716c4dd49c7b292016a118002d1, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:32.896565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f7d8dd-579f-4ed6-a5e5-1a3ca2d42047 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:32.906621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1183/2379] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:52:32 2022] PUT /identity/v3/projects/40cec83c2c47499bb24fe889373dafe3/users/19d02716c4dd49c7b292016a118002d1/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:52:32.917177 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d260e-d8ae-4231-8d7e-aacf296993cc None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:32.917470 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d260e-d8ae-4231-8d7e-aacf296993cc None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:32.917753 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d260e-d8ae-4231-8d7e-aacf296993cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:32.942969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-763d260e-d8ae-4231-8d7e-aacf296993cc None None] MFA Rules not processed for user `19d02716c4dd49c7b292016a118002d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:32.944189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-763d260e-d8ae-4231-8d7e-aacf296993cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:33.010853 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1197/2380] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:32 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:33.021802 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7d8dd-579f-4ed6-a5e5-1a3ca2d42047 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:33.023512 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d3f7d8dd-579f-4ed6-a5e5-1a3ca2d42047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:33.080598 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7d8dd-579f-4ed6-a5e5-1a3ca2d42047 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:33.085183 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5fa98-4859-441a-921e-e4d063df867c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:33.085815 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5fa98-4859-441a-921e-e4d063df867c service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:33.086096 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5fa98-4859-441a-921e-e4d063df867c service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:33.086369 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5fa98-4859-441a-921e-e4d063df867c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:33.095163 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b5fa98-4859-441a-921e-e4d063df867c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:33.097672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b5fa98-4859-441a-921e-e4d063df867c service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:33.107093 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1184/2381] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:33 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:52:34.539492 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d260e-d8ae-4231-8d7e-aacf296993cc None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:34.541275 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-763d260e-d8ae-4231-8d7e-aacf296993cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:34.590611 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:34.610195 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d260e-d8ae-4231-8d7e-aacf296993cc None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:34.614932 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737bab7e-74cb-4c39-ba2c-7dcc6b3d17d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:34.615638 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bab7e-74cb-4c39-ba2c-7dcc6b3d17d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:34.615918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bab7e-74cb-4c39-ba2c-7dcc6b3d17d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:34.616190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bab7e-74cb-4c39-ba2c-7dcc6b3d17d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:34.618468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-737bab7e-74cb-4c39-ba2c-7dcc6b3d17d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:34.676486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737bab7e-74cb-4c39-ba2c-7dcc6b3d17d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:34.678046 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737bab7e-74cb-4c39-ba2c-7dcc6b3d17d8 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:34.687278 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1198/2382] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:52:35.929170 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5fa98-4859-441a-921e-e4d063df867c service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:36.001287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:36.025708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5fa98-4859-441a-921e-e4d063df867c service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:36.031314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a710e7-5ed4-4d07-943d-6b973520e596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:36.032451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a710e7-5ed4-4d07-943d-6b973520e596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:36.032885 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a710e7-5ed4-4d07-943d-6b973520e596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:36.033301 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a710e7-5ed4-4d07-943d-6b973520e596 admin admin] PATH_INFO: `/v3/users/ae1923f6c730415aa0e3be9e0fa07dac` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:36.035776 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a710e7-5ed4-4d07-943d-6b973520e596 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae1923f6c730415aa0e3be9e0fa07dac)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:36.037912 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a710e7-5ed4-4d07-943d-6b973520e596 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:36.227802 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c8a710e7-5ed4-4d07-943d-6b973520e596 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae1923f6c730415aa0e3be9e0fa07dac'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:36.240057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c8a710e7-5ed4-4d07-943d-6b973520e596 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae1923f6c730415aa0e3be9e0fa07dac'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:36.241699 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c8a710e7-5ed4-4d07-943d-6b973520e596 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae1923f6c730415aa0e3be9e0fa07dac'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:36.242751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c8a710e7-5ed4-4d07-943d-6b973520e596 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae1923f6c730415aa0e3be9e0fa07dac'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:36.268020 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c8a710e7-5ed4-4d07-943d-6b973520e596 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae1923f6c730415aa0e3be9e0fa07dac'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:36.362890 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1185/2383] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:52:35 2022] DELETE /identity/v3/users/ae1923f6c730415aa0e3be9e0fa07dac => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:36.367457 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a64a8-d617-4d19-bc22-18a550e16014 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:36.368340 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a64a8-d617-4d19-bc22-18a550e16014 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:36.368621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a64a8-d617-4d19-bc22-18a550e16014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:36.389041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c65a64a8-d617-4d19-bc22-18a550e16014 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:36.390393 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c65a64a8-d617-4d19-bc22-18a550e16014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:36.459550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1199/2384] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:36.469734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a710e7-5ed4-4d07-943d-6b973520e596 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:36.471180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c8a710e7-5ed4-4d07-943d-6b973520e596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:36.536690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a710e7-5ed4-4d07-943d-6b973520e596 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:36.540407 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c81a3-49a2-41cc-8cf2-85c5b558c322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:36.541140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c81a3-49a2-41cc-8cf2-85c5b558c322 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:36.541497 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c81a3-49a2-41cc-8cf2-85c5b558c322 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:36.541775 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c81a3-49a2-41cc-8cf2-85c5b558c322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:36.551707 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c81a3-49a2-41cc-8cf2-85c5b558c322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:36.553154 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c81a3-49a2-41cc-8cf2-85c5b558c322 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:36.562789 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1186/2385] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:52:36.964015 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a64a8-d617-4d19-bc22-18a550e16014 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:36.965526 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c65a64a8-d617-4d19-bc22-18a550e16014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:36.978137 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a64a8-d617-4d19-bc22-18a550e16014 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:36.982645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d1c9d-cfc6-4ffe-a7a3-893fc75ba87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:36.983499 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d1c9d-cfc6-4ffe-a7a3-893fc75ba87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:36.983780 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d1c9d-cfc6-4ffe-a7a3-893fc75ba87c admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:36.984056 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d1c9d-cfc6-4ffe-a7a3-893fc75ba87c admin admin] PATH_INFO: `/v3/projects/a3307c176a7747878d7e48600d69522b` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:36.985240 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d1c9d-cfc6-4ffe-a7a3-893fc75ba87c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3307c176a7747878d7e48600d69522b)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:36.987799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d1c9d-cfc6-4ffe-a7a3-893fc75ba87c admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:37.091694 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d80d1c9d-cfc6-4ffe-a7a3-893fc75ba87c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3307c176a7747878d7e48600d69522b', 'request_id': 'req-d80d1c9d-cfc6-4ffe-a7a3-893fc75ba87c'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:37.098862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d80d1c9d-cfc6-4ffe-a7a3-893fc75ba87c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3307c176a7747878d7e48600d69522b', 'request_id': 'req-d80d1c9d-cfc6-4ffe-a7a3-893fc75ba87c'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:37.161952 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d80d1c9d-cfc6-4ffe-a7a3-893fc75ba87c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3307c176a7747878d7e48600d69522b', 'request_id': 'req-d80d1c9d-cfc6-4ffe-a7a3-893fc75ba87c'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:37.172565 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d80d1c9d-cfc6-4ffe-a7a3-893fc75ba87c admin admin] The token cache is being invalidate because project a3307c176a7747878d7e48600d69522b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:52:37.172969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d80d1c9d-cfc6-4ffe-a7a3-893fc75ba87c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:37.173502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d80d1c9d-cfc6-4ffe-a7a3-893fc75ba87c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:37.175750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1200/2386] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:52:36 2022] DELETE /identity/v3/projects/a3307c176a7747878d7e48600d69522b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:37.179741 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c81a3-49a2-41cc-8cf2-85c5b558c322 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:37.189509 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c81a3-49a2-41cc-8cf2-85c5b558c322 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:37.245792 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f908552-0bd3-456d-9cc1-75697184f1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:37.246691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f908552-0bd3-456d-9cc1-75697184f1d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:37.246974 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f908552-0bd3-456d-9cc1-75697184f1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:37.247248 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f908552-0bd3-456d-9cc1-75697184f1d0 admin admin] PATH_INFO: `/v3/users/19d02716c4dd49c7b292016a118002d1` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:37.248610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f908552-0bd3-456d-9cc1-75697184f1d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19d02716c4dd49c7b292016a118002d1)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:37.250598 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f908552-0bd3-456d-9cc1-75697184f1d0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:37.252100 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:37.424784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2f908552-0bd3-456d-9cc1-75697184f1d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19d02716c4dd49c7b292016a118002d1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:37.432588 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2f908552-0bd3-456d-9cc1-75697184f1d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19d02716c4dd49c7b292016a118002d1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:37.433306 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2f908552-0bd3-456d-9cc1-75697184f1d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19d02716c4dd49c7b292016a118002d1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:37.433932 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2f908552-0bd3-456d-9cc1-75697184f1d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19d02716c4dd49c7b292016a118002d1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:37.446971 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2f908552-0bd3-456d-9cc1-75697184f1d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19d02716c4dd49c7b292016a118002d1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:37.534874 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1187/2387] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:52:37 2022] DELETE /identity/v3/users/19d02716c4dd49c7b292016a118002d1 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:52:37.984558 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d1c9d-cfc6-4ffe-a7a3-893fc75ba87c admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:37.988215 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d80d1c9d-cfc6-4ffe-a7a3-893fc75ba87c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:37.999935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d1c9d-cfc6-4ffe-a7a3-893fc75ba87c admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:38.014402 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:38.062172 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c53d8f-33ca-412d-98e1-fc801ad89da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:38.063076 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c53d8f-33ca-412d-98e1-fc801ad89da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:38.063360 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c53d8f-33ca-412d-98e1-fc801ad89da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:38.063636 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c53d8f-33ca-412d-98e1-fc801ad89da7 admin admin] PATH_INFO: `/v3/projects/40cec83c2c47499bb24fe889373dafe3` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:38.064823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c53d8f-33ca-412d-98e1-fc801ad89da7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40cec83c2c47499bb24fe889373dafe3)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:38.067051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c53d8f-33ca-412d-98e1-fc801ad89da7 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:38.194738 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-42c53d8f-33ca-412d-98e1-fc801ad89da7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40cec83c2c47499bb24fe889373dafe3', 'request_id': 'req-42c53d8f-33ca-412d-98e1-fc801ad89da7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:38.200712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-42c53d8f-33ca-412d-98e1-fc801ad89da7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40cec83c2c47499bb24fe889373dafe3', 'request_id': 'req-42c53d8f-33ca-412d-98e1-fc801ad89da7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:38.269538 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-42c53d8f-33ca-412d-98e1-fc801ad89da7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40cec83c2c47499bb24fe889373dafe3', 'request_id': 'req-42c53d8f-33ca-412d-98e1-fc801ad89da7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:38.281495 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-42c53d8f-33ca-412d-98e1-fc801ad89da7 admin admin] The token cache is being invalidate because project 40cec83c2c47499bb24fe889373dafe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:52:38.281786 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-42c53d8f-33ca-412d-98e1-fc801ad89da7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:38.282437 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-42c53d8f-33ca-412d-98e1-fc801ad89da7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:38.284101 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1201/2388] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:52:37 2022] DELETE /identity/v3/projects/40cec83c2c47499bb24fe889373dafe3 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:38.288943 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd2bef-c302-425a-b657-f22970bd8160 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:38.289455 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd2bef-c302-425a-b657-f22970bd8160 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:38.289565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd2bef-c302-425a-b657-f22970bd8160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:38.308157 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d6dd2bef-c302-425a-b657-f22970bd8160 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:38.309365 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d6dd2bef-c302-425a-b657-f22970bd8160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:38.369124 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1188/2389] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:38.378824 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1a5ba-4bde-4995-bcd1-2bfdd40bfe03 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:38.379034 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1a5ba-4bde-4995-bcd1-2bfdd40bfe03 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:38.379308 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1a5ba-4bde-4995-bcd1-2bfdd40bfe03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:38.397413 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-7af1a5ba-4bde-4995-bcd1-2bfdd40bfe03 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:38.398641 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-7af1a5ba-4bde-4995-bcd1-2bfdd40bfe03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:38.406523 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1202/2390] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:38.411090 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd2bef-c302-425a-b657-f22970bd8160 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:38.421369 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd2bef-c302-425a-b657-f22970bd8160 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:38.427343 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117d293-c404-456d-990e-1b8e810ec387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:38.427977 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117d293-c404-456d-990e-1b8e810ec387 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:38.428297 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117d293-c404-456d-990e-1b8e810ec387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:38.428576 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117d293-c404-456d-990e-1b8e810ec387 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:38.429306 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117d293-c404-456d-990e-1b8e810ec387 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:52:38.429608 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117d293-c404-456d-990e-1b8e810ec387 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:38.431240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117d293-c404-456d-990e-1b8e810ec387 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:38.445329 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1189/2391] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:52:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:52:38.454464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1a5ba-4bde-4995-bcd1-2bfdd40bfe03 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:38.466277 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1a5ba-4bde-4995-bcd1-2bfdd40bfe03 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:38.476302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d16bb1-2439-4d98-b67d-3a493b5cf581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:38.477002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d16bb1-2439-4d98-b67d-3a493b5cf581 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:38.477246 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d16bb1-2439-4d98-b67d-3a493b5cf581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:38.477692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d16bb1-2439-4d98-b67d-3a493b5cf581 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:38.478363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d16bb1-2439-4d98-b67d-3a493b5cf581 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:38.481056 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d16bb1-2439-4d98-b67d-3a493b5cf581 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:38.631617 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1203/2392] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:52:38 2022] POST /identity/v3/projects => generated 385 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:52:38.635803 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117d293-c404-456d-990e-1b8e810ec387 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:38.646877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117d293-c404-456d-990e-1b8e810ec387 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:38.704851 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d353b-cdbe-4c40-9ad3-aa637e6639f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:38.705504 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d353b-cdbe-4c40-9ad3-aa637e6639f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:38.706826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d353b-cdbe-4c40-9ad3-aa637e6639f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:38.707115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d353b-cdbe-4c40-9ad3-aa637e6639f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:38.707816 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d353b-cdbe-4c40-9ad3-aa637e6639f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:38.709725 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d353b-cdbe-4c40-9ad3-aa637e6639f8 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:38.719223 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:38.806566 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1190/2393] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:52:38 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:52:38.808441 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d16bb1-2439-4d98-b67d-3a493b5cf581 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:38.821439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d16bb1-2439-4d98-b67d-3a493b5cf581 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:38.826329 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106946e-a1f7-4ac7-abbe-c9b71b3731c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:38.827047 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106946e-a1f7-4ac7-abbe-c9b71b3731c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:38.827326 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106946e-a1f7-4ac7-abbe-c9b71b3731c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:38.827594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106946e-a1f7-4ac7-abbe-c9b71b3731c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:38.828323 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106946e-a1f7-4ac7-abbe-c9b71b3731c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:38.829915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106946e-a1f7-4ac7-abbe-c9b71b3731c9 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:38.845178 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1204/2394] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:38.846480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d353b-cdbe-4c40-9ad3-aa637e6639f8 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:38.857483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d353b-cdbe-4c40-9ad3-aa637e6639f8 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:38.865110 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ff7a0-9f29-43c6-bb9a-a7a1aa3f5ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:38.865110 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ff7a0-9f29-43c6-bb9a-a7a1aa3f5ecd admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:38.865110 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ff7a0-9f29-43c6-bb9a-a7a1aa3f5ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:38.865110 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ff7a0-9f29-43c6-bb9a-a7a1aa3f5ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:38.865500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ff7a0-9f29-43c6-bb9a-a7a1aa3f5ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:38.866646 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ff7a0-9f29-43c6-bb9a-a7a1aa3f5ecd admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:38.878913 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1191/2395] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:38.883373 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106946e-a1f7-4ac7-abbe-c9b71b3731c9 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:38.893805 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106946e-a1f7-4ac7-abbe-c9b71b3731c9 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:38.898456 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3454ea0-4a22-408b-919a-9757f253e001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:38.901251 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3454ea0-4a22-408b-919a-9757f253e001 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:38.901533 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3454ea0-4a22-408b-919a-9757f253e001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:38.901813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3454ea0-4a22-408b-919a-9757f253e001 admin admin] PATH_INFO: `/v3/projects/392beae929bc4ea6b98d6b09444d1474/users/df332ec36b754f489da706b132118a7b/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:38.919159 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3454ea0-4a22-408b-919a-9757f253e001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=392beae929bc4ea6b98d6b09444d1474, user_id=df332ec36b754f489da706b132118a7b, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:38.922685 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3454ea0-4a22-408b-919a-9757f253e001 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:39.009483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1205/2396] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:52:38 2022] PUT /identity/v3/projects/392beae929bc4ea6b98d6b09444d1474/users/df332ec36b754f489da706b132118a7b/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:39.020006 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe4b91-505e-45d8-a78c-66c56c252116 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:39.020340 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe4b91-505e-45d8-a78c-66c56c252116 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:39.020618 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe4b91-505e-45d8-a78c-66c56c252116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:39.047759 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-49fe4b91-505e-45d8-a78c-66c56c252116 None None] MFA Rules not processed for user `df332ec36b754f489da706b132118a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:39.048876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-49fe4b91-505e-45d8-a78c-66c56c252116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:39.115148 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1192/2397] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:39.119726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3454ea0-4a22-408b-919a-9757f253e001 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:39.165639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:39.183316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3454ea0-4a22-408b-919a-9757f253e001 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:39.187926 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41145d-7ba5-43ca-a77a-12bb2d27f2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:39.188585 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41145d-7ba5-43ca-a77a-12bb2d27f2b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:39.188868 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41145d-7ba5-43ca-a77a-12bb2d27f2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:39.189140 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41145d-7ba5-43ca-a77a-12bb2d27f2b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:39.189863 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41145d-7ba5-43ca-a77a-12bb2d27f2b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:39.192524 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41145d-7ba5-43ca-a77a-12bb2d27f2b9 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:39.295095 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1206/2398] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:52:39 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:52:39.299138 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe4b91-505e-45d8-a78c-66c56c252116 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:39.310232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe4b91-505e-45d8-a78c-66c56c252116 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:39.366732 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ffb64-495d-4730-bdb7-311466fda032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:39.367401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ffb64-495d-4730-bdb7-311466fda032 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:39.367683 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ffb64-495d-4730-bdb7-311466fda032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:39.367958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ffb64-495d-4730-bdb7-311466fda032 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:39.368677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ffb64-495d-4730-bdb7-311466fda032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:39.370669 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ffb64-495d-4730-bdb7-311466fda032 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:39.390001 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41145d-7ba5-43ca-a77a-12bb2d27f2b9 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:39.392408 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-6c41145d-7ba5-43ca-a77a-12bb2d27f2b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:39.468481 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1193/2399] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:52:39 2022] POST /identity/v3/users => generated 299 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:52:39.468718 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41145d-7ba5-43ca-a77a-12bb2d27f2b9 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:39.469736 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ffb64-495d-4730-bdb7-311466fda032 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:39.470902 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664c1de-4b0b-4f7c-9c0f-da27e16a2411 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac04b32777ca4f929e9e59deee6e0260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:39.472071 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664c1de-4b0b-4f7c-9c0f-da27e16a2411 service placement] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:39.472352 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664c1de-4b0b-4f7c-9c0f-da27e16a2411 service placement] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:39.472632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664c1de-4b0b-4f7c-9c0f-da27e16a2411 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:39.474687 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d664c1de-4b0b-4f7c-9c0f-da27e16a2411 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:39.480845 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ffb64-495d-4730-bdb7-311466fda032 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:39.486182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3217aa-be88-42a2-8e67-d0721142d9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:39.486456 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3217aa-be88-42a2-8e67-d0721142d9c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:39.486833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3217aa-be88-42a2-8e67-d0721142d9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:39.486973 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3217aa-be88-42a2-8e67-d0721142d9c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:39.488416 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3217aa-be88-42a2-8e67-d0721142d9c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:39.490036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3217aa-be88-42a2-8e67-d0721142d9c2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:39.493101 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664c1de-4b0b-4f7c-9c0f-da27e16a2411 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:39.495927 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664c1de-4b0b-4f7c-9c0f-da27e16a2411 service placement] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:39.500501 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1194/2400] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:39.505081 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3217aa-be88-42a2-8e67-d0721142d9c2 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:39.507199 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1207/2401] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:52:39.515395 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3217aa-be88-42a2-8e67-d0721142d9c2 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:39.520229 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff18fb-e290-4758-bdeb-33a349db742b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:39.521108 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff18fb-e290-4758-bdeb-33a349db742b admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:39.521392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff18fb-e290-4758-bdeb-33a349db742b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:39.521668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff18fb-e290-4758-bdeb-33a349db742b admin admin] PATH_INFO: `/v3/projects/c1f0c30d162e4ee6870d13dd297e08ae/users/0bed9942b6794c69a5484a23f15901b0/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:39.536949 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff18fb-e290-4758-bdeb-33a349db742b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f0c30d162e4ee6870d13dd297e08ae, user_id=0bed9942b6794c69a5484a23f15901b0, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:39.539659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff18fb-e290-4758-bdeb-33a349db742b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:39.644637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1195/2402] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:52:39 2022] PUT /identity/v3/projects/c1f0c30d162e4ee6870d13dd297e08ae/users/0bed9942b6794c69a5484a23f15901b0/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:52:39.648531 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664c1de-4b0b-4f7c-9c0f-da27e16a2411 service placement] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:39.659304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664c1de-4b0b-4f7c-9c0f-da27e16a2411 service placement] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:39.726320 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2660227-78b3-416c-bb8c-c48fff9ca19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:39.727024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2660227-78b3-416c-bb8c-c48fff9ca19c admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:39.727311 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2660227-78b3-416c-bb8c-c48fff9ca19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:39.727591 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2660227-78b3-416c-bb8c-c48fff9ca19c admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:39.728310 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2660227-78b3-416c-bb8c-c48fff9ca19c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:39.729906 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2660227-78b3-416c-bb8c-c48fff9ca19c admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:39.740932 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1208/2403] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:39.745241 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff18fb-e290-4758-bdeb-33a349db742b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:39.756251 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff18fb-e290-4758-bdeb-33a349db742b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:39.760923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae33a6-5790-4f68-9dcf-262dd732197a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:39.761798 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae33a6-5790-4f68-9dcf-262dd732197a admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:39.762077 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae33a6-5790-4f68-9dcf-262dd732197a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:39.762350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae33a6-5790-4f68-9dcf-262dd732197a admin admin] PATH_INFO: `/v3/projects/c1f0c30d162e4ee6870d13dd297e08ae/users/0bed9942b6794c69a5484a23f15901b0/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:39.764549 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae33a6-5790-4f68-9dcf-262dd732197a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f0c30d162e4ee6870d13dd297e08ae, user_id=0bed9942b6794c69a5484a23f15901b0, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:39.767291 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae33a6-5790-4f68-9dcf-262dd732197a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:39.777714 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1196/2404] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:52:39 2022] PUT /identity/v3/projects/c1f0c30d162e4ee6870d13dd297e08ae/users/0bed9942b6794c69a5484a23f15901b0/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:39.787948 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bdd37-41ff-4e22-8b4d-d84983654ec9 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:39.788245 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bdd37-41ff-4e22-8b4d-d84983654ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:39.788528 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bdd37-41ff-4e22-8b4d-d84983654ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:39.816411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-a58bdd37-41ff-4e22-8b4d-d84983654ec9 None None] MFA Rules not processed for user `0bed9942b6794c69a5484a23f15901b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:39.817564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a58bdd37-41ff-4e22-8b4d-d84983654ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:39.858088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:39.886967 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1209/2405] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:39.898235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae33a6-5790-4f68-9dcf-262dd732197a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:39.900023 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-11ae33a6-5790-4f68-9dcf-262dd732197a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:39.920644 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:39.958812 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae33a6-5790-4f68-9dcf-262dd732197a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:39.962384 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41fceb-757b-4a28-b29a-cc613bf127bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:39.963061 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41fceb-757b-4a28-b29a-cc613bf127bb service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:39.963411 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41fceb-757b-4a28-b29a-cc613bf127bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:39.963640 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41fceb-757b-4a28-b29a-cc613bf127bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:39.973296 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41fceb-757b-4a28-b29a-cc613bf127bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:39.975041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41fceb-757b-4a28-b29a-cc613bf127bb service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:39.984389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1197/2406] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:52:40.013141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58bdd37-41ff-4e22-8b4d-d84983654ec9 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:40.074906 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58bdd37-41ff-4e22-8b4d-d84983654ec9 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:40.079540 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1265676-8282-43f6-9f42-66ac6274293e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:40.080502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1265676-8282-43f6-9f42-66ac6274293e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:40.080663 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1265676-8282-43f6-9f42-66ac6274293e admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:40.081056 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1265676-8282-43f6-9f42-66ac6274293e admin admin] PATH_INFO: `/v3/users/df332ec36b754f489da706b132118a7b` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:40.082722 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1265676-8282-43f6-9f42-66ac6274293e admin admin] RBAC: Authorizing `identity:delete_user(user_id=df332ec36b754f489da706b132118a7b)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:40.084632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1265676-8282-43f6-9f42-66ac6274293e admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:40.283278 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-a1265676-8282-43f6-9f42-66ac6274293e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df332ec36b754f489da706b132118a7b'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:40.291061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-a1265676-8282-43f6-9f42-66ac6274293e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df332ec36b754f489da706b132118a7b'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:40.291769 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-a1265676-8282-43f6-9f42-66ac6274293e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df332ec36b754f489da706b132118a7b'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:40.292603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-a1265676-8282-43f6-9f42-66ac6274293e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df332ec36b754f489da706b132118a7b'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:40.304973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-a1265676-8282-43f6-9f42-66ac6274293e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df332ec36b754f489da706b132118a7b'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:40.367780 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1210/2407] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:52:40 2022] DELETE /identity/v3/users/df332ec36b754f489da706b132118a7b => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:40.372520 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafbdae-702a-4276-a1d6-84a8c5543d57 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:40.372980 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafbdae-702a-4276-a1d6-84a8c5543d57 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:40.373102 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafbdae-702a-4276-a1d6-84a8c5543d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:40.390755 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-faafbdae-702a-4276-a1d6-84a8c5543d57 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:40.391992 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-faafbdae-702a-4276-a1d6-84a8c5543d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:40.453512 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1198/2408] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:40.463945 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1265676-8282-43f6-9f42-66ac6274293e admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:40.465357 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a1265676-8282-43f6-9f42-66ac6274293e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:40.523035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1265676-8282-43f6-9f42-66ac6274293e admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:40.526661 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015eea98-aacd-4755-a794-fa5a19b51710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:40.527304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015eea98-aacd-4755-a794-fa5a19b51710 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:40.527587 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015eea98-aacd-4755-a794-fa5a19b51710 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:40.527862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015eea98-aacd-4755-a794-fa5a19b51710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:40.537644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015eea98-aacd-4755-a794-fa5a19b51710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:40.539117 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015eea98-aacd-4755-a794-fa5a19b51710 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:40.543995 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:40.548652 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1211/2409] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:52:40.974751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafbdae-702a-4276-a1d6-84a8c5543d57 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:40.975909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-faafbdae-702a-4276-a1d6-84a8c5543d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:40.988250 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafbdae-702a-4276-a1d6-84a8c5543d57 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:40.993058 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d408b-9fd5-4a47-9b10-813ebb923d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:40.993960 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d408b-9fd5-4a47-9b10-813ebb923d1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:40.994391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d408b-9fd5-4a47-9b10-813ebb923d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:40.994712 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d408b-9fd5-4a47-9b10-813ebb923d1a admin admin] PATH_INFO: `/v3/projects/392beae929bc4ea6b98d6b09444d1474` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:40.996654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d408b-9fd5-4a47-9b10-813ebb923d1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=392beae929bc4ea6b98d6b09444d1474)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:40.998710 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d408b-9fd5-4a47-9b10-813ebb923d1a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:41.065539 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015eea98-aacd-4755-a794-fa5a19b51710 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:41.067215 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-015eea98-aacd-4755-a794-fa5a19b51710 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:41.078698 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015eea98-aacd-4755-a794-fa5a19b51710 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:41.083122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d375e3-1b80-4186-b9ab-f619cb71ea83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:41.085166 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d375e3-1b80-4186-b9ab-f619cb71ea83 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:41.085166 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d375e3-1b80-4186-b9ab-f619cb71ea83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:41.085166 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d375e3-1b80-4186-b9ab-f619cb71ea83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:41.086130 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c8d375e3-1b80-4186-b9ab-f619cb71ea83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:41.098443 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d375e3-1b80-4186-b9ab-f619cb71ea83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:41.100197 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d375e3-1b80-4186-b9ab-f619cb71ea83 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:41.114201 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1212/2410] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:52:41.126070 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ce2d408b-9fd5-4a47-9b10-813ebb923d1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '392beae929bc4ea6b98d6b09444d1474', 'request_id': 'req-ce2d408b-9fd5-4a47-9b10-813ebb923d1a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:41.133623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ce2d408b-9fd5-4a47-9b10-813ebb923d1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '392beae929bc4ea6b98d6b09444d1474', 'request_id': 'req-ce2d408b-9fd5-4a47-9b10-813ebb923d1a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:41.192082 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ce2d408b-9fd5-4a47-9b10-813ebb923d1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '392beae929bc4ea6b98d6b09444d1474', 'request_id': 'req-ce2d408b-9fd5-4a47-9b10-813ebb923d1a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:41.203082 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ce2d408b-9fd5-4a47-9b10-813ebb923d1a admin admin] The token cache is being invalidate because project 392beae929bc4ea6b98d6b09444d1474 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:52:41.203370 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ce2d408b-9fd5-4a47-9b10-813ebb923d1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:41.204010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ce2d408b-9fd5-4a47-9b10-813ebb923d1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:41.205623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1199/2411] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:52:40 2022] DELETE /identity/v3/projects/392beae929bc4ea6b98d6b09444d1474 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:41.210121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d375e3-1b80-4186-b9ab-f619cb71ea83 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:41.219917 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d375e3-1b80-4186-b9ab-f619cb71ea83 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:41.281367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e1f6d-5f24-45b9-bdcc-a34cc1c451a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:41.282332 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e1f6d-5f24-45b9-bdcc-a34cc1c451a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:41.282652 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e1f6d-5f24-45b9-bdcc-a34cc1c451a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:41.282933 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e1f6d-5f24-45b9-bdcc-a34cc1c451a6 admin admin] PATH_INFO: `/v3/users/0bed9942b6794c69a5484a23f15901b0` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:41.284473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e1f6d-5f24-45b9-bdcc-a34cc1c451a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bed9942b6794c69a5484a23f15901b0)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:41.286462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e1f6d-5f24-45b9-bdcc-a34cc1c451a6 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:41.454011 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-933e1f6d-5f24-45b9-bdcc-a34cc1c451a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bed9942b6794c69a5484a23f15901b0'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:41.461897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-933e1f6d-5f24-45b9-bdcc-a34cc1c451a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bed9942b6794c69a5484a23f15901b0'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:41.462623 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-933e1f6d-5f24-45b9-bdcc-a34cc1c451a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bed9942b6794c69a5484a23f15901b0'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:41.463248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-933e1f6d-5f24-45b9-bdcc-a34cc1c451a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bed9942b6794c69a5484a23f15901b0'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:41.477151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-933e1f6d-5f24-45b9-bdcc-a34cc1c451a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bed9942b6794c69a5484a23f15901b0'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:41.552134 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1213/2412] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:52:41 2022] DELETE /identity/v3/users/0bed9942b6794c69a5484a23f15901b0 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:42.085176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d408b-9fd5-4a47-9b10-813ebb923d1a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:42.086835 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ce2d408b-9fd5-4a47-9b10-813ebb923d1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:42.125642 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d408b-9fd5-4a47-9b10-813ebb923d1a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:42.178802 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:42.195760 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6cae6-f313-4451-bba5-f7bd211b9651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:42.196646 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6cae6-f313-4451-bba5-f7bd211b9651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:42.197296 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6cae6-f313-4451-bba5-f7bd211b9651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:42.197296 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6cae6-f313-4451-bba5-f7bd211b9651 admin admin] PATH_INFO: `/v3/projects/c1f0c30d162e4ee6870d13dd297e08ae` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:42.198430 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd6cae6-f313-4451-bba5-f7bd211b9651 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1f0c30d162e4ee6870d13dd297e08ae)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:42.200449 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd6cae6-f313-4451-bba5-f7bd211b9651 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:42.288419 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-dfd6cae6-f313-4451-bba5-f7bd211b9651 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1f0c30d162e4ee6870d13dd297e08ae', 'request_id': 'req-dfd6cae6-f313-4451-bba5-f7bd211b9651'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:42.294791 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-dfd6cae6-f313-4451-bba5-f7bd211b9651 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1f0c30d162e4ee6870d13dd297e08ae', 'request_id': 'req-dfd6cae6-f313-4451-bba5-f7bd211b9651'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:42.351118 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-dfd6cae6-f313-4451-bba5-f7bd211b9651 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1f0c30d162e4ee6870d13dd297e08ae', 'request_id': 'req-dfd6cae6-f313-4451-bba5-f7bd211b9651'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:42.363502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-dfd6cae6-f313-4451-bba5-f7bd211b9651 admin admin] The token cache is being invalidate because project c1f0c30d162e4ee6870d13dd297e08ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:52:42.363805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-dfd6cae6-f313-4451-bba5-f7bd211b9651 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:42.364599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-dfd6cae6-f313-4451-bba5-f7bd211b9651 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:42.366384 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1200/2413] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:52:42 2022] DELETE /identity/v3/projects/c1f0c30d162e4ee6870d13dd297e08ae => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:42.371273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58261e78-697c-408b-b541-41f97bef5bd3 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:42.371729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58261e78-697c-408b-b541-41f97bef5bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:42.371977 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58261e78-697c-408b-b541-41f97bef5bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:42.392665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-58261e78-697c-408b-b541-41f97bef5bd3 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:42.394684 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-58261e78-697c-408b-b541-41f97bef5bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:42.454911 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1214/2414] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:42.465091 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f34204-f5ae-41c8-87a2-6eea9a953b8f None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:42.465383 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f34204-f5ae-41c8-87a2-6eea9a953b8f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:42.465663 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f34204-f5ae-41c8-87a2-6eea9a953b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:42.485276 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-12f34204-f5ae-41c8-87a2-6eea9a953b8f None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:42.486490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-12f34204-f5ae-41c8-87a2-6eea9a953b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:42.495251 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1201/2415] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:42.499359 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58261e78-697c-408b-b541-41f97bef5bd3 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:42.510136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58261e78-697c-408b-b541-41f97bef5bd3 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:42.515114 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b20767-e87c-4229-839c-26538dcb6795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:42.520821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b20767-e87c-4229-839c-26538dcb6795 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:42.520821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b20767-e87c-4229-839c-26538dcb6795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:42.520821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b20767-e87c-4229-839c-26538dcb6795 admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:42.520821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b20767-e87c-4229-839c-26538dcb6795 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:52:42.520821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b20767-e87c-4229-839c-26538dcb6795 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:42.520821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b20767-e87c-4229-839c-26538dcb6795 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:42.523009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:42.539209 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1215/2416] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:52:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:52:42.540390 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f34204-f5ae-41c8-87a2-6eea9a953b8f None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:42.550670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f34204-f5ae-41c8-87a2-6eea9a953b8f None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:42.555558 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393069ee-a191-4fd4-9f29-2e713814b970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:42.556218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393069ee-a191-4fd4-9f29-2e713814b970 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:42.556518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393069ee-a191-4fd4-9f29-2e713814b970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:42.556932 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393069ee-a191-4fd4-9f29-2e713814b970 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:42.559946 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393069ee-a191-4fd4-9f29-2e713814b970 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:42.559946 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393069ee-a191-4fd4-9f29-2e713814b970 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:42.684201 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1202/2417] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:52:42 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:52:42.688581 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b20767-e87c-4229-839c-26538dcb6795 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:42.700112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b20767-e87c-4229-839c-26538dcb6795 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:42.771844 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463bcdae-5d2e-49ac-aba9-cdbaf19aaf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:42.772922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463bcdae-5d2e-49ac-aba9-cdbaf19aaf2d admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:42.772922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463bcdae-5d2e-49ac-aba9-cdbaf19aaf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:42.773355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463bcdae-5d2e-49ac-aba9-cdbaf19aaf2d admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:42.774305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463bcdae-5d2e-49ac-aba9-cdbaf19aaf2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:42.776670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463bcdae-5d2e-49ac-aba9-cdbaf19aaf2d admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:42.855276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1216/2418] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:52:42 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:52:42.859489 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393069ee-a191-4fd4-9f29-2e713814b970 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:42.872450 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393069ee-a191-4fd4-9f29-2e713814b970 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:42.879090 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677dcd1-ff2e-4fe7-a354-3ba8aabdbb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:42.879440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677dcd1-ff2e-4fe7-a354-3ba8aabdbb34 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:42.879723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677dcd1-ff2e-4fe7-a354-3ba8aabdbb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:42.879994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677dcd1-ff2e-4fe7-a354-3ba8aabdbb34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:42.880960 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b677dcd1-ff2e-4fe7-a354-3ba8aabdbb34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:42.882588 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b677dcd1-ff2e-4fe7-a354-3ba8aabdbb34 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:42.895199 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1203/2419] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:42.899010 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463bcdae-5d2e-49ac-aba9-cdbaf19aaf2d admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:42.909161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463bcdae-5d2e-49ac-aba9-cdbaf19aaf2d admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:42.913934 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac592e-7087-44d3-9de6-bef391e5b6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:42.914627 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac592e-7087-44d3-9de6-bef391e5b6a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:42.915220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac592e-7087-44d3-9de6-bef391e5b6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:42.915220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac592e-7087-44d3-9de6-bef391e5b6a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:42.915896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac592e-7087-44d3-9de6-bef391e5b6a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:42.917752 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac592e-7087-44d3-9de6-bef391e5b6a8 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:42.932636 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1217/2420] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:42.940719 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677dcd1-ff2e-4fe7-a354-3ba8aabdbb34 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:42.952526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677dcd1-ff2e-4fe7-a354-3ba8aabdbb34 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:42.957952 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409f583-05de-498e-a85a-20598a36659d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:42.958988 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409f583-05de-498e-a85a-20598a36659d admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:42.959346 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409f583-05de-498e-a85a-20598a36659d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:42.959696 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409f583-05de-498e-a85a-20598a36659d admin admin] PATH_INFO: `/v3/projects/6fc1b8c435914114bcf368df57fc446c/users/b8bcf1abeab24b15b507ddaa5d66910e/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:42.978015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c409f583-05de-498e-a85a-20598a36659d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fc1b8c435914114bcf368df57fc446c, user_id=b8bcf1abeab24b15b507ddaa5d66910e, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:42.980849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c409f583-05de-498e-a85a-20598a36659d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:43.048656 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1204/2421] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:52:42 2022] PUT /identity/v3/projects/6fc1b8c435914114bcf368df57fc446c/users/b8bcf1abeab24b15b507ddaa5d66910e/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:43.059080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178585f-281c-4d55-b6ef-124a4feeb9c7 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:43.059458 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178585f-281c-4d55-b6ef-124a4feeb9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:43.059736 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178585f-281c-4d55-b6ef-124a4feeb9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:43.090935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-b178585f-281c-4d55-b6ef-124a4feeb9c7 None None] MFA Rules not processed for user `b8bcf1abeab24b15b507ddaa5d66910e`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:43.092224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b178585f-281c-4d55-b6ef-124a4feeb9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:43.170981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1218/2422] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:43.181986 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409f583-05de-498e-a85a-20598a36659d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:43.183796 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c409f583-05de-498e-a85a-20598a36659d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:43.265405 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409f583-05de-498e-a85a-20598a36659d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:43.270160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a1413d-9632-417e-b984-22d07d4e6264 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:43.270844 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a1413d-9632-417e-b984-22d07d4e6264 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:43.271124 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a1413d-9632-417e-b984-22d07d4e6264 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:43.271404 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a1413d-9632-417e-b984-22d07d4e6264 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:43.279734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a1413d-9632-417e-b984-22d07d4e6264 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:43.281153 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a1413d-9632-417e-b984-22d07d4e6264 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:43.290455 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1205/2423] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:52:48.370019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b178585f-281c-4d55-b6ef-124a4feeb9c7 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:48.371646 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b178585f-281c-4d55-b6ef-124a4feeb9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:48.445030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:48.449897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b178585f-281c-4d55-b6ef-124a4feeb9c7 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:48.455044 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d89c15-c27a-4524-9bd9-391da9f21ee6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac04b32777ca4f929e9e59deee6e0260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:48.455715 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d89c15-c27a-4524-9bd9-391da9f21ee6 service placement] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:48.457021 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d89c15-c27a-4524-9bd9-391da9f21ee6 service placement] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:48.457021 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d89c15-c27a-4524-9bd9-391da9f21ee6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:48.459474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-51d89c15-c27a-4524-9bd9-391da9f21ee6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:48.471856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d89c15-c27a-4524-9bd9-391da9f21ee6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:48.473474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d89c15-c27a-4524-9bd9-391da9f21ee6 service placement] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:48.485721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1219/2424] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:52:48.609971 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a1413d-9632-417e-b984-22d07d4e6264 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:48.611678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-16a1413d-9632-417e-b984-22d07d4e6264 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:48.628975 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:48.678462 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a1413d-9632-417e-b984-22d07d4e6264 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:48.683247 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfaf39-0e0b-4437-82eb-268dee4a1b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:48.684117 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfaf39-0e0b-4437-82eb-268dee4a1b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:48.684400 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfaf39-0e0b-4437-82eb-268dee4a1b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:48.684672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfaf39-0e0b-4437-82eb-268dee4a1b35 admin admin] PATH_INFO: `/v3/users/626c3dbdc0f1407db82d6988e68e40b2` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:48.685943 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfaf39-0e0b-4437-82eb-268dee4a1b35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=626c3dbdc0f1407db82d6988e68e40b2)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:48.687903 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfaf39-0e0b-4437-82eb-268dee4a1b35 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:48.926678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9cdfaf39-0e0b-4437-82eb-268dee4a1b35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '626c3dbdc0f1407db82d6988e68e40b2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:48.934621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9cdfaf39-0e0b-4437-82eb-268dee4a1b35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '626c3dbdc0f1407db82d6988e68e40b2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:48.935323 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9cdfaf39-0e0b-4437-82eb-268dee4a1b35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '626c3dbdc0f1407db82d6988e68e40b2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:48.935937 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9cdfaf39-0e0b-4437-82eb-268dee4a1b35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '626c3dbdc0f1407db82d6988e68e40b2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:48.948473 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9cdfaf39-0e0b-4437-82eb-268dee4a1b35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '626c3dbdc0f1407db82d6988e68e40b2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:49.041301 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1206/2425] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:52:48 2022] DELETE /identity/v3/users/626c3dbdc0f1407db82d6988e68e40b2 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:49.299045 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d89c15-c27a-4524-9bd9-391da9f21ee6 service placement] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:49.312572 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d89c15-c27a-4524-9bd9-391da9f21ee6 service placement] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:49.382490 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c79051-1ad5-45a2-814b-98a3a79cfdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:49.383312 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c79051-1ad5-45a2-814b-98a3a79cfdc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:49.383591 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c79051-1ad5-45a2-814b-98a3a79cfdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:49.383861 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c79051-1ad5-45a2-814b-98a3a79cfdc7 admin admin] PATH_INFO: `/v3/projects/ce01d9a78d3f4bb7b101b47548b73037` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:49.385033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c79051-1ad5-45a2-814b-98a3a79cfdc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce01d9a78d3f4bb7b101b47548b73037)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:49.386959 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c79051-1ad5-45a2-814b-98a3a79cfdc7 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:49.483321 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f7c79051-1ad5-45a2-814b-98a3a79cfdc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce01d9a78d3f4bb7b101b47548b73037', 'request_id': 'req-f7c79051-1ad5-45a2-814b-98a3a79cfdc7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:49.490628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f7c79051-1ad5-45a2-814b-98a3a79cfdc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce01d9a78d3f4bb7b101b47548b73037', 'request_id': 'req-f7c79051-1ad5-45a2-814b-98a3a79cfdc7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:49.548855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f7c79051-1ad5-45a2-814b-98a3a79cfdc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce01d9a78d3f4bb7b101b47548b73037', 'request_id': 'req-f7c79051-1ad5-45a2-814b-98a3a79cfdc7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:49.560176 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f7c79051-1ad5-45a2-814b-98a3a79cfdc7 admin admin] The token cache is being invalidate because project ce01d9a78d3f4bb7b101b47548b73037 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:52:49.560525 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f7c79051-1ad5-45a2-814b-98a3a79cfdc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:49.561197 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f7c79051-1ad5-45a2-814b-98a3a79cfdc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:49.562780 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1220/2426] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:52:49 2022] DELETE /identity/v3/projects/ce01d9a78d3f4bb7b101b47548b73037 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:49.567435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfaf39-0e0b-4437-82eb-268dee4a1b35 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:49.568954 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9cdfaf39-0e0b-4437-82eb-268dee4a1b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:49.582157 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfaf39-0e0b-4437-82eb-268dee4a1b35 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:49.644882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ff1e5-351b-45cf-b4d4-d3079310ed3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:49.644882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ff1e5-351b-45cf-b4d4-d3079310ed3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:49.644882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ff1e5-351b-45cf-b4d4-d3079310ed3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:49.645707 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ff1e5-351b-45cf-b4d4-d3079310ed3e admin admin] PATH_INFO: `/v3/users/c3a61b9ccab24c6882917e33812beead` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:49.646344 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ff1e5-351b-45cf-b4d4-d3079310ed3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3a61b9ccab24c6882917e33812beead)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:49.648320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ff1e5-351b-45cf-b4d4-d3079310ed3e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:49.807389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:49.820176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5e5ff1e5-351b-45cf-b4d4-d3079310ed3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3a61b9ccab24c6882917e33812beead'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:49.827131 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5e5ff1e5-351b-45cf-b4d4-d3079310ed3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3a61b9ccab24c6882917e33812beead'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:49.827807 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5e5ff1e5-351b-45cf-b4d4-d3079310ed3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3a61b9ccab24c6882917e33812beead'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:49.828417 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5e5ff1e5-351b-45cf-b4d4-d3079310ed3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3a61b9ccab24c6882917e33812beead'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:49.840230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5e5ff1e5-351b-45cf-b4d4-d3079310ed3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3a61b9ccab24c6882917e33812beead'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:49.926944 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1207/2427] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:52:49 2022] DELETE /identity/v3/users/c3a61b9ccab24c6882917e33812beead => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:50.175578 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c79051-1ad5-45a2-814b-98a3a79cfdc7 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:50.179070 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f7c79051-1ad5-45a2-814b-98a3a79cfdc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:50.190684 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c79051-1ad5-45a2-814b-98a3a79cfdc7 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:50.248620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831509a-ba24-49a1-b38f-39b87df24179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:50.249482 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831509a-ba24-49a1-b38f-39b87df24179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:50.249764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831509a-ba24-49a1-b38f-39b87df24179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:50.250039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831509a-ba24-49a1-b38f-39b87df24179 admin admin] PATH_INFO: `/v3/projects/28c51273c8554c95ab585206bf920f3e` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:50.251221 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831509a-ba24-49a1-b38f-39b87df24179 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28c51273c8554c95ab585206bf920f3e)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:50.253455 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831509a-ba24-49a1-b38f-39b87df24179 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:50.277811 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:50.373688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-e831509a-ba24-49a1-b38f-39b87df24179 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28c51273c8554c95ab585206bf920f3e', 'request_id': 'req-e831509a-ba24-49a1-b38f-39b87df24179'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:50.380250 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-e831509a-ba24-49a1-b38f-39b87df24179 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28c51273c8554c95ab585206bf920f3e', 'request_id': 'req-e831509a-ba24-49a1-b38f-39b87df24179'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:50.459640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-e831509a-ba24-49a1-b38f-39b87df24179 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28c51273c8554c95ab585206bf920f3e', 'request_id': 'req-e831509a-ba24-49a1-b38f-39b87df24179'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:50.471093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-e831509a-ba24-49a1-b38f-39b87df24179 admin admin] The token cache is being invalidate because project 28c51273c8554c95ab585206bf920f3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:52:50.471406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-e831509a-ba24-49a1-b38f-39b87df24179 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:50.472703 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-e831509a-ba24-49a1-b38f-39b87df24179 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:50.475027 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1221/2428] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:52:50 2022] DELETE /identity/v3/projects/28c51273c8554c95ab585206bf920f3e => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:54.397808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ff1e5-351b-45cf-b4d4-d3079310ed3e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:54.399310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5e5ff1e5-351b-45cf-b4d4-d3079310ed3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:54.412862 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ff1e5-351b-45cf-b4d4-d3079310ed3e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:54.473908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bc9f1-95ba-4649-943d-8460faba96e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:54.475112 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bc9f1-95ba-4649-943d-8460faba96e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:54.475627 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bc9f1-95ba-4649-943d-8460faba96e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:54.475903 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bc9f1-95ba-4649-943d-8460faba96e8 admin admin] PATH_INFO: `/v3/users/b8bcf1abeab24b15b507ddaa5d66910e` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:54.477437 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074bc9f1-95ba-4649-943d-8460faba96e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8bcf1abeab24b15b507ddaa5d66910e)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:54.479726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074bc9f1-95ba-4649-943d-8460faba96e8 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:54.639264 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-074bc9f1-95ba-4649-943d-8460faba96e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8bcf1abeab24b15b507ddaa5d66910e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:54.646098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-074bc9f1-95ba-4649-943d-8460faba96e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8bcf1abeab24b15b507ddaa5d66910e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:54.646818 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-074bc9f1-95ba-4649-943d-8460faba96e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8bcf1abeab24b15b507ddaa5d66910e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:54.647428 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-074bc9f1-95ba-4649-943d-8460faba96e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8bcf1abeab24b15b507ddaa5d66910e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:54.659407 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-074bc9f1-95ba-4649-943d-8460faba96e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8bcf1abeab24b15b507ddaa5d66910e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:54.750327 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1208/2429] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:52:54 2022] DELETE /identity/v3/users/b8bcf1abeab24b15b507ddaa5d66910e => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:54.755497 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8351fc-90ae-4a0a-9b17-50b26d48279d None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:54.756005 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8351fc-90ae-4a0a-9b17-50b26d48279d None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:54.756140 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8351fc-90ae-4a0a-9b17-50b26d48279d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:54.773461 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-fb8351fc-90ae-4a0a-9b17-50b26d48279d None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:54.775036 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-fb8351fc-90ae-4a0a-9b17-50b26d48279d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:54.840519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1222/2430] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:54.850936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bc9f1-95ba-4649-943d-8460faba96e8 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:54.852369 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-074bc9f1-95ba-4649-943d-8460faba96e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:54.884470 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:54.909749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bc9f1-95ba-4649-943d-8460faba96e8 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:54.912208 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73200ecc-6094-4730-a7c7-62dc107c924b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:54.912853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73200ecc-6094-4730-a7c7-62dc107c924b service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:54.913136 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73200ecc-6094-4730-a7c7-62dc107c924b service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:54.913410 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73200ecc-6094-4730-a7c7-62dc107c924b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:54.922458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73200ecc-6094-4730-a7c7-62dc107c924b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:54.923916 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73200ecc-6094-4730-a7c7-62dc107c924b service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:54.934625 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1209/2431] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:52:55.356525 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8351fc-90ae-4a0a-9b17-50b26d48279d None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:55.357946 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-fb8351fc-90ae-4a0a-9b17-50b26d48279d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:55.371900 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8351fc-90ae-4a0a-9b17-50b26d48279d None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:55.376845 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53a68e-7f08-41f2-8ba4-5ef81bcaa0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:55.377816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53a68e-7f08-41f2-8ba4-5ef81bcaa0a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:55.378098 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53a68e-7f08-41f2-8ba4-5ef81bcaa0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:55.378378 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53a68e-7f08-41f2-8ba4-5ef81bcaa0a3 admin admin] PATH_INFO: `/v3/projects/6fc1b8c435914114bcf368df57fc446c` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:55.379607 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53a68e-7f08-41f2-8ba4-5ef81bcaa0a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fc1b8c435914114bcf368df57fc446c)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:55.381536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53a68e-7f08-41f2-8ba4-5ef81bcaa0a3 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:55.481384 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fa53a68e-7f08-41f2-8ba4-5ef81bcaa0a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fc1b8c435914114bcf368df57fc446c', 'request_id': 'req-fa53a68e-7f08-41f2-8ba4-5ef81bcaa0a3'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:55.487546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fa53a68e-7f08-41f2-8ba4-5ef81bcaa0a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fc1b8c435914114bcf368df57fc446c', 'request_id': 'req-fa53a68e-7f08-41f2-8ba4-5ef81bcaa0a3'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:55.556765 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fa53a68e-7f08-41f2-8ba4-5ef81bcaa0a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fc1b8c435914114bcf368df57fc446c', 'request_id': 'req-fa53a68e-7f08-41f2-8ba4-5ef81bcaa0a3'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:55.567740 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fa53a68e-7f08-41f2-8ba4-5ef81bcaa0a3 admin admin] The token cache is being invalidate because project 6fc1b8c435914114bcf368df57fc446c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:52:55.568032 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fa53a68e-7f08-41f2-8ba4-5ef81bcaa0a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:55.568720 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fa53a68e-7f08-41f2-8ba4-5ef81bcaa0a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:52:55.570329 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1223/2432] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:52:55 2022] DELETE /identity/v3/projects/6fc1b8c435914114bcf368df57fc446c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:55.575166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bac626-fe75-4efa-a717-a22037f3ad4a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:55.579086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bac626-fe75-4efa-a717-a22037f3ad4a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:55.579086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bac626-fe75-4efa-a717-a22037f3ad4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:55.593518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-35bac626-fe75-4efa-a717-a22037f3ad4a None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:55.594763 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-35bac626-fe75-4efa-a717-a22037f3ad4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:55.653785 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1210/2433] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:55.662372 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11db9d-8d30-4729-ae41-73e3f8a90e22 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:55.662694 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11db9d-8d30-4729-ae41-73e3f8a90e22 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:55.662972 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11db9d-8d30-4729-ae41-73e3f8a90e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:55.685350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-ac11db9d-8d30-4729-ae41-73e3f8a90e22 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:55.686548 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-ac11db9d-8d30-4729-ae41-73e3f8a90e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:55.694447 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1224/2434] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:55.698804 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bac626-fe75-4efa-a717-a22037f3ad4a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:55.709052 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bac626-fe75-4efa-a717-a22037f3ad4a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:55.713838 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583a401-cbed-4e51-b43e-52a1f5f42784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:55.714489 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583a401-cbed-4e51-b43e-52a1f5f42784 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:55.714803 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583a401-cbed-4e51-b43e-52a1f5f42784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:55.715075 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583a401-cbed-4e51-b43e-52a1f5f42784 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:55.715790 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583a401-cbed-4e51-b43e-52a1f5f42784 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:52:55.716095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583a401-cbed-4e51-b43e-52a1f5f42784 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:55.717678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583a401-cbed-4e51-b43e-52a1f5f42784 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:55.730012 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1211/2435] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:52:55.734400 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11db9d-8d30-4729-ae41-73e3f8a90e22 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:55.743447 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11db9d-8d30-4729-ae41-73e3f8a90e22 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:55.748055 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff89e67-8a6d-4332-81c3-ff4c150d3905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:55.748688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff89e67-8a6d-4332-81c3-ff4c150d3905 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:55.748968 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff89e67-8a6d-4332-81c3-ff4c150d3905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:55.749236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff89e67-8a6d-4332-81c3-ff4c150d3905 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:55.749958 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff89e67-8a6d-4332-81c3-ff4c150d3905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:55.751880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff89e67-8a6d-4332-81c3-ff4c150d3905 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:55.837898 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1225/2436] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:52:55 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:52:55.842044 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583a401-cbed-4e51-b43e-52a1f5f42784 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:55.852487 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583a401-cbed-4e51-b43e-52a1f5f42784 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:55.908718 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45a409-9e72-4200-b994-b42a79c176b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:55.910166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45a409-9e72-4200-b994-b42a79c176b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:55.910486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45a409-9e72-4200-b994-b42a79c176b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:55.910806 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45a409-9e72-4200-b994-b42a79c176b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:55.911537 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b45a409-9e72-4200-b994-b42a79c176b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:55.913480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b45a409-9e72-4200-b994-b42a79c176b2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:55.973653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1212/2437] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:52:55 2022] POST /identity/v3/users => generated 285 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:52:55.978090 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff89e67-8a6d-4332-81c3-ff4c150d3905 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:55.988927 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff89e67-8a6d-4332-81c3-ff4c150d3905 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:55.993353 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702e67d-2677-41ab-97fc-8cf7dc3eb0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:55.993998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702e67d-2677-41ab-97fc-8cf7dc3eb0be admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:55.994277 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702e67d-2677-41ab-97fc-8cf7dc3eb0be admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:55.994572 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702e67d-2677-41ab-97fc-8cf7dc3eb0be admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:55.995289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e702e67d-2677-41ab-97fc-8cf7dc3eb0be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:55.996899 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e702e67d-2677-41ab-97fc-8cf7dc3eb0be admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:55.998131 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:56.008442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1226/2438] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:56.012441 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45a409-9e72-4200-b994-b42a79c176b2 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:56.023063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45a409-9e72-4200-b994-b42a79c176b2 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:56.027978 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36322dd9-0bfb-42bc-a9fb-a8a4a21511b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:56.032056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36322dd9-0bfb-42bc-a9fb-a8a4a21511b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:56.032056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36322dd9-0bfb-42bc-a9fb-a8a4a21511b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:56.032056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36322dd9-0bfb-42bc-a9fb-a8a4a21511b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:56.032056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36322dd9-0bfb-42bc-a9fb-a8a4a21511b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:56.032056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36322dd9-0bfb-42bc-a9fb-a8a4a21511b4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:56.041509 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1213/2439] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:52:56.045543 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702e67d-2677-41ab-97fc-8cf7dc3eb0be admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:56.055765 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702e67d-2677-41ab-97fc-8cf7dc3eb0be admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:56.060332 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba74611-e2f0-49f2-b2d6-1e165a36496e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:56.061202 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba74611-e2f0-49f2-b2d6-1e165a36496e admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:56.061482 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba74611-e2f0-49f2-b2d6-1e165a36496e admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:56.061761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba74611-e2f0-49f2-b2d6-1e165a36496e admin admin] PATH_INFO: `/v3/projects/43ae9248838d4a6bb01bb0984b8a3586/users/6da15a3d0e454483bf6f27db42c86f27/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:56.077647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba74611-e2f0-49f2-b2d6-1e165a36496e admin admin] RBAC: Authorizing `identity:create_grant(project_id=43ae9248838d4a6bb01bb0984b8a3586, user_id=6da15a3d0e454483bf6f27db42c86f27, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:56.080406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba74611-e2f0-49f2-b2d6-1e165a36496e admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:56.149358 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1227/2440] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:52:56 2022] PUT /identity/v3/projects/43ae9248838d4a6bb01bb0984b8a3586/users/6da15a3d0e454483bf6f27db42c86f27/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:52:56.215125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfba4a-5d76-4f99-aa65-f26be2191382 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:56.215420 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfba4a-5d76-4f99-aa65-f26be2191382 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:56.215702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfba4a-5d76-4f99-aa65-f26be2191382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:56.242873 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-01cfba4a-5d76-4f99-aa65-f26be2191382 None None] MFA Rules not processed for user `6da15a3d0e454483bf6f27db42c86f27`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:56.244007 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-01cfba4a-5d76-4f99-aa65-f26be2191382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:56.270378 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:56.309750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1214/2441] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:52:56 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:56.325682 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba74611-e2f0-49f2-b2d6-1e165a36496e admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:56.325682 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-1ba74611-e2f0-49f2-b2d6-1e165a36496e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:56.382787 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba74611-e2f0-49f2-b2d6-1e165a36496e admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:56.386493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaeec6-4258-47a4-b396-9fc67eb4341d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6105838cbf1409587ee7d9e3be554f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:56.387195 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaeec6-4258-47a4-b396-9fc67eb4341d service glance] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:56.387481 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaeec6-4258-47a4-b396-9fc67eb4341d service glance] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:56.387758 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaeec6-4258-47a4-b396-9fc67eb4341d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:56.396302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaeec6-4258-47a4-b396-9fc67eb4341d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:56.397764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaeec6-4258-47a4-b396-9fc67eb4341d service glance] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:56.406733 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1228/2442] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:56 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:53:06.373688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=15 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:53:06.402641 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=38,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:53:12.667068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bfe5d-869f-4599-ab27-8302fffc059e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:12.670741 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bfe5d-869f-4599-ab27-8302fffc059e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:12.670741 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bfe5d-869f-4599-ab27-8302fffc059e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:12.685539 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-949bfe5d-869f-4599-ab27-8302fffc059e None None] MFA Rules not processed for user `b7fd8323d29d445f903e2103453e8ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:53:12.686772 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-949bfe5d-869f-4599-ab27-8302fffc059e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:53:12.741260 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1215/2443] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:53:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:53:22.734614 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=25 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:53:39.303103 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaeec6-4258-47a4-b396-9fc67eb4341d service glance] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:53:39.304528 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d7aaeec6-4258-47a4-b396-9fc67eb4341d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:53:39.368689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaeec6-4258-47a4-b396-9fc67eb4341d service glance] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:53:39.368689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d2696-d5be-4858-a514-a242d651341e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:53:39.369190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d2696-d5be-4858-a514-a242d651341e service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:39.369474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d2696-d5be-4858-a514-a242d651341e service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:39.369744 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d2696-d5be-4858-a514-a242d651341e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:39.371652 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e16d2696-d5be-4858-a514-a242d651341e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:53:39.438495 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d2696-d5be-4858-a514-a242d651341e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:53:39.440192 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d2696-d5be-4858-a514-a242d651341e service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:53:39.449231 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1229/2444] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:53:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:53:46.064410 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949bfe5d-869f-4599-ab27-8302fffc059e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:53:46.124803 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949bfe5d-869f-4599-ab27-8302fffc059e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:53:46.129808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ac221-e22c-4b03-bba7-5a0609a22627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:53:46.130694 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ac221-e22c-4b03-bba7-5a0609a22627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:46.130975 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ac221-e22c-4b03-bba7-5a0609a22627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:46.131245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ac221-e22c-4b03-bba7-5a0609a22627 admin admin] PATH_INFO: `/v3/users/6da15a3d0e454483bf6f27db42c86f27` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:46.132397 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ac221-e22c-4b03-bba7-5a0609a22627 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6da15a3d0e454483bf6f27db42c86f27)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:53:46.134328 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ac221-e22c-4b03-bba7-5a0609a22627 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:53:46.312829 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b06ac221-e22c-4b03-bba7-5a0609a22627 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6da15a3d0e454483bf6f27db42c86f27'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:53:46.319774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b06ac221-e22c-4b03-bba7-5a0609a22627 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6da15a3d0e454483bf6f27db42c86f27'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:53:46.320429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b06ac221-e22c-4b03-bba7-5a0609a22627 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6da15a3d0e454483bf6f27db42c86f27'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:53:46.321077 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b06ac221-e22c-4b03-bba7-5a0609a22627 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6da15a3d0e454483bf6f27db42c86f27'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:53:46.332741 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b06ac221-e22c-4b03-bba7-5a0609a22627 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6da15a3d0e454483bf6f27db42c86f27'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:53:46.413825 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1216/2445] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:53:46 2022] DELETE /identity/v3/users/6da15a3d0e454483bf6f27db42c86f27 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:53:46.418359 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ec08d-e40d-4862-88fe-b0b4f278269d None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:46.418876 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ec08d-e40d-4862-88fe-b0b4f278269d None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:46.418976 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ec08d-e40d-4862-88fe-b0b4f278269d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:46.437110 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-b81ec08d-e40d-4862-88fe-b0b4f278269d None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:53:46.438296 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b81ec08d-e40d-4862-88fe-b0b4f278269d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:53:46.503165 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1230/2446] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:53:46.512706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ac221-e22c-4b03-bba7-5a0609a22627 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:53:46.514110 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b06ac221-e22c-4b03-bba7-5a0609a22627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:53:46.571175 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ac221-e22c-4b03-bba7-5a0609a22627 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:53:46.574902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29508f-d15a-44ba-938a-cd1a0bda45d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:53:46.575552 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29508f-d15a-44ba-938a-cd1a0bda45d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:46.575838 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29508f-d15a-44ba-938a-cd1a0bda45d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:46.576118 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29508f-d15a-44ba-938a-cd1a0bda45d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:46.585666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a29508f-d15a-44ba-938a-cd1a0bda45d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:53:46.587188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a29508f-d15a-44ba-938a-cd1a0bda45d0 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:53:46.597178 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1217/2447] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:53:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:53:47.038366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ec08d-e40d-4862-88fe-b0b4f278269d None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:53:47.039857 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b81ec08d-e40d-4862-88fe-b0b4f278269d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:53:47.052007 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ec08d-e40d-4862-88fe-b0b4f278269d None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:53:47.057050 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb071dd-7459-4ba8-853e-d2e5326dd3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:53:47.057980 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb071dd-7459-4ba8-853e-d2e5326dd3a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:47.058235 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb071dd-7459-4ba8-853e-d2e5326dd3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:47.058511 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb071dd-7459-4ba8-853e-d2e5326dd3a2 admin admin] PATH_INFO: `/v3/projects/43ae9248838d4a6bb01bb0984b8a3586` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:47.059721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb071dd-7459-4ba8-853e-d2e5326dd3a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43ae9248838d4a6bb01bb0984b8a3586)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:53:47.061622 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb071dd-7459-4ba8-853e-d2e5326dd3a2 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:53:47.168860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4bb071dd-7459-4ba8-853e-d2e5326dd3a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43ae9248838d4a6bb01bb0984b8a3586', 'request_id': 'req-4bb071dd-7459-4ba8-853e-d2e5326dd3a2'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:53:47.169977 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:53:47.176079 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4bb071dd-7459-4ba8-853e-d2e5326dd3a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43ae9248838d4a6bb01bb0984b8a3586', 'request_id': 'req-4bb071dd-7459-4ba8-853e-d2e5326dd3a2'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:53:47.246644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4bb071dd-7459-4ba8-853e-d2e5326dd3a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43ae9248838d4a6bb01bb0984b8a3586', 'request_id': 'req-4bb071dd-7459-4ba8-853e-d2e5326dd3a2'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:53:47.257358 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4bb071dd-7459-4ba8-853e-d2e5326dd3a2 admin admin] The token cache is being invalidate because project 43ae9248838d4a6bb01bb0984b8a3586 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:53:47.257643 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4bb071dd-7459-4ba8-853e-d2e5326dd3a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:53:47.258347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4bb071dd-7459-4ba8-853e-d2e5326dd3a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:53:47.260001 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1231/2448] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:53:47 2022] DELETE /identity/v3/projects/43ae9248838d4a6bb01bb0984b8a3586 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:53:47.265073 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3ebd3-85e2-4a5d-b69a-063bf10e7935 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:47.265569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3ebd3-85e2-4a5d-b69a-063bf10e7935 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:47.265667 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3ebd3-85e2-4a5d-b69a-063bf10e7935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:47.284206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-58d3ebd3-85e2-4a5d-b69a-063bf10e7935 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:53:47.285453 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-58d3ebd3-85e2-4a5d-b69a-063bf10e7935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:53:47.301526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:53:47.343427 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1218/2449] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:53:47.352370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c4759-bdf5-4b1c-ab6f-6bb2829700d6 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:47.352663 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c4759-bdf5-4b1c-ab6f-6bb2829700d6 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:47.352947 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c4759-bdf5-4b1c-ab6f-6bb2829700d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:47.370815 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-e20c4759-bdf5-4b1c-ab6f-6bb2829700d6 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:53:47.372007 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e20c4759-bdf5-4b1c-ab6f-6bb2829700d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:53:47.380037 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1232/2450] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:53:47.384133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3ebd3-85e2-4a5d-b69a-063bf10e7935 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:53:47.394523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3ebd3-85e2-4a5d-b69a-063bf10e7935 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:53:47.399027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8d968-2db5-4e44-85e7-52cdf26b3a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:53:47.399665 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8d968-2db5-4e44-85e7-52cdf26b3a94 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:47.399951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8d968-2db5-4e44-85e7-52cdf26b3a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:47.400247 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8d968-2db5-4e44-85e7-52cdf26b3a94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:47.400999 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e8d968-2db5-4e44-85e7-52cdf26b3a94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:53:47.401514 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e8d968-2db5-4e44-85e7-52cdf26b3a94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:53:47.403681 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e8d968-2db5-4e44-85e7-52cdf26b3a94 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:53:47.415934 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1219/2451] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:53:47.420160 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c4759-bdf5-4b1c-ab6f-6bb2829700d6 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:53:47.429685 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c4759-bdf5-4b1c-ab6f-6bb2829700d6 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:53:47.434283 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f6d99-e6c8-4143-9eaf-0a52d753b397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:53:47.434968 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f6d99-e6c8-4143-9eaf-0a52d753b397 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:47.435343 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f6d99-e6c8-4143-9eaf-0a52d753b397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:47.435623 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f6d99-e6c8-4143-9eaf-0a52d753b397 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:47.436310 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f6d99-e6c8-4143-9eaf-0a52d753b397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:53:47.438224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f6d99-e6c8-4143-9eaf-0a52d753b397 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:53:47.535807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1233/2452] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:53:47 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:53:47.539964 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8d968-2db5-4e44-85e7-52cdf26b3a94 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:53:47.550125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8d968-2db5-4e44-85e7-52cdf26b3a94 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:53:47.605528 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a31fff-9b56-4e58-9d9c-0483c6574628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:53:47.606204 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a31fff-9b56-4e58-9d9c-0483c6574628 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:47.606473 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a31fff-9b56-4e58-9d9c-0483c6574628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:47.606777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a31fff-9b56-4e58-9d9c-0483c6574628 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:47.607474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a31fff-9b56-4e58-9d9c-0483c6574628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:53:47.609392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a31fff-9b56-4e58-9d9c-0483c6574628 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:53:47.706181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1220/2453] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:53:47 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:53:47.710350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f6d99-e6c8-4143-9eaf-0a52d753b397 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:53:47.720954 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f6d99-e6c8-4143-9eaf-0a52d753b397 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:53:47.725456 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d4f11-ba30-445e-9cdf-f7f7736233c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:53:47.726358 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d4f11-ba30-445e-9cdf-f7f7736233c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:47.726670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d4f11-ba30-445e-9cdf-f7f7736233c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:47.726947 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d4f11-ba30-445e-9cdf-f7f7736233c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:47.727665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d4f11-ba30-445e-9cdf-f7f7736233c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:53:47.729244 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d4f11-ba30-445e-9cdf-f7f7736233c7 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:53:47.740027 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1234/2454] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:53:47.743597 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a31fff-9b56-4e58-9d9c-0483c6574628 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:53:47.754896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a31fff-9b56-4e58-9d9c-0483c6574628 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:53:47.759572 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1a4c0-11cc-432c-b3ea-4d4539a5c877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:53:47.760223 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1a4c0-11cc-432c-b3ea-4d4539a5c877 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:47.760507 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1a4c0-11cc-432c-b3ea-4d4539a5c877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:47.760786 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1a4c0-11cc-432c-b3ea-4d4539a5c877 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:47.761526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1a4c0-11cc-432c-b3ea-4d4539a5c877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:53:47.763138 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1a4c0-11cc-432c-b3ea-4d4539a5c877 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:53:47.773514 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1221/2455] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:53:47.777640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d4f11-ba30-445e-9cdf-f7f7736233c7 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:53:47.787588 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d4f11-ba30-445e-9cdf-f7f7736233c7 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:53:47.792136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436110d0-a25e-4613-be15-07e92c69edde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:53:47.793003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436110d0-a25e-4613-be15-07e92c69edde admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:47.793284 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436110d0-a25e-4613-be15-07e92c69edde admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:47.793560 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436110d0-a25e-4613-be15-07e92c69edde admin admin] PATH_INFO: `/v3/projects/17a1542e108d4e3e85fe5525a7e331d0/users/15a4c8e15cd44904805c2519c4b6864a/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:47.808608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436110d0-a25e-4613-be15-07e92c69edde admin admin] RBAC: Authorizing `identity:create_grant(project_id=17a1542e108d4e3e85fe5525a7e331d0, user_id=15a4c8e15cd44904805c2519c4b6864a, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:53:47.811310 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436110d0-a25e-4613-be15-07e92c69edde admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:53:47.894387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1235/2456] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:53:47 2022] PUT /identity/v3/projects/17a1542e108d4e3e85fe5525a7e331d0/users/15a4c8e15cd44904805c2519c4b6864a/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:53:47.899443 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1530c-d08b-4f78-88ae-a5c7bcecb88b None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:47.899743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1530c-d08b-4f78-88ae-a5c7bcecb88b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:47.900029 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1530c-d08b-4f78-88ae-a5c7bcecb88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:47.918140 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-1cf1530c-d08b-4f78-88ae-a5c7bcecb88b None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:53:47.919399 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1cf1530c-d08b-4f78-88ae-a5c7bcecb88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:53:47.978742 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1222/2457] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:53:47.988838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436110d0-a25e-4613-be15-07e92c69edde admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:53:47.990756 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-436110d0-a25e-4613-be15-07e92c69edde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:53:48.049086 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436110d0-a25e-4613-be15-07e92c69edde admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:53:48.052782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0c733-517f-4ffc-a8f8-6f911581c073 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:53:48.053412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0c733-517f-4ffc-a8f8-6f911581c073 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:48.053696 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0c733-517f-4ffc-a8f8-6f911581c073 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:48.053972 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0c733-517f-4ffc-a8f8-6f911581c073 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:48.062729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0c733-517f-4ffc-a8f8-6f911581c073 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:53:48.064744 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0c733-517f-4ffc-a8f8-6f911581c073 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:53:48.074803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1236/2458] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:53:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:53:52.120502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38991d-fa36-4881-9131-57b15e29a426 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:52.121631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38991d-fa36-4881-9131-57b15e29a426 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:52.122090 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38991d-fa36-4881-9131-57b15e29a426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:52.139983 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ad38991d-fa36-4881-9131-57b15e29a426 None None] MFA Rules not processed for user `6cc5a4676f6a416cbdc8c67ebd80cdf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:53:52.145320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ad38991d-fa36-4881-9131-57b15e29a426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:53:52.166208 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:53:52.204783 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1223/2459] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:53:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:53:53.630204 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c953dcd-72bf-4cf6-a552-90e2d431dfa4 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:53.631202 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c953dcd-72bf-4cf6-a552-90e2d431dfa4 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:53.631682 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c953dcd-72bf-4cf6-a552-90e2d431dfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:53.696469 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-5c953dcd-72bf-4cf6-a552-90e2d431dfa4 None None] MFA Rules not processed for user `15a4c8e15cd44904805c2519c4b6864a`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:53:53.697994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-5c953dcd-72bf-4cf6-a552-90e2d431dfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:53:53.803635 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1237/2460] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:53:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:53:53.805101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38991d-fa36-4881-9131-57b15e29a426 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:53:53.819509 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38991d-fa36-4881-9131-57b15e29a426 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:53:53.824447 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e931c8e-0ed4-4757-9893-fd298cbd88b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:53:53.825122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e931c8e-0ed4-4757-9893-fd298cbd88b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:53.825436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e931c8e-0ed4-4757-9893-fd298cbd88b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:53.825658 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e931c8e-0ed4-4757-9893-fd298cbd88b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:53.826610 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e931c8e-0ed4-4757-9893-fd298cbd88b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:53:53.839237 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e931c8e-0ed4-4757-9893-fd298cbd88b2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:53:53.960330 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1224/2461] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:53:53 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:53:53.962374 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c953dcd-72bf-4cf6-a552-90e2d431dfa4 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:53:53.973637 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c953dcd-72bf-4cf6-a552-90e2d431dfa4 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:53:53.977636 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:53:54.039205 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f327d-be23-45c1-90da-b35814914837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:53:54.039834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f327d-be23-45c1-90da-b35814914837 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:54.040121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f327d-be23-45c1-90da-b35814914837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:54.040399 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f327d-be23-45c1-90da-b35814914837 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:54.041196 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03f327d-be23-45c1-90da-b35814914837 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:53:54.043152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03f327d-be23-45c1-90da-b35814914837 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:53:54.139939 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1238/2462] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:53:53 2022] POST /identity/v3/users => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:53:54.144144 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e931c8e-0ed4-4757-9893-fd298cbd88b2 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:53:54.156142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e931c8e-0ed4-4757-9893-fd298cbd88b2 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:53:54.162048 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c0a9a-0563-4f88-8149-8eafd315019c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:53:54.162745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c0a9a-0563-4f88-8149-8eafd315019c admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:54.163031 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c0a9a-0563-4f88-8149-8eafd315019c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:54.163309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c0a9a-0563-4f88-8149-8eafd315019c admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:54.164040 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c0a9a-0563-4f88-8149-8eafd315019c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:53:54.166418 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c0a9a-0563-4f88-8149-8eafd315019c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:53:54.177762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1225/2463] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:53:54.183775 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f327d-be23-45c1-90da-b35814914837 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:53:54.196242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f327d-be23-45c1-90da-b35814914837 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:53:54.202256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a6a5c-746f-4cf0-a105-4ec844bff9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:53:54.203809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a6a5c-746f-4cf0-a105-4ec844bff9cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:54.204092 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a6a5c-746f-4cf0-a105-4ec844bff9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:54.204368 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a6a5c-746f-4cf0-a105-4ec844bff9cd admin admin] PATH_INFO: `/v3/projects/5fdf048be460453d937439116caa2f4f/users/dbfc00a8fc9f4b1f9535619c5a957c81/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:54.222923 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a6a5c-746f-4cf0-a105-4ec844bff9cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fdf048be460453d937439116caa2f4f, user_id=dbfc00a8fc9f4b1f9535619c5a957c81, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:53:54.225701 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a6a5c-746f-4cf0-a105-4ec844bff9cd admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:53:54.343606 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1239/2464] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:53:54 2022] PUT /identity/v3/projects/5fdf048be460453d937439116caa2f4f/users/dbfc00a8fc9f4b1f9535619c5a957c81/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:53:54.348601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c0a9a-0563-4f88-8149-8eafd315019c admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:53:54.359319 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c0a9a-0563-4f88-8149-8eafd315019c admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:53:54.421897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00440a-cf18-4f97-bcfc-99db26f6071a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:53:54.422635 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00440a-cf18-4f97-bcfc-99db26f6071a admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:54.422916 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00440a-cf18-4f97-bcfc-99db26f6071a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:54.423240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00440a-cf18-4f97-bcfc-99db26f6071a admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:54.424060 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00440a-cf18-4f97-bcfc-99db26f6071a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:53:54.426723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00440a-cf18-4f97-bcfc-99db26f6071a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:53:54.439302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1226/2465] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:53:54.447755 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a6a5c-746f-4cf0-a105-4ec844bff9cd admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:53:54.462600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a6a5c-746f-4cf0-a105-4ec844bff9cd admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:53:54.467828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22f8b6-9b5f-44c8-81f5-9f3bfe7e4fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:53:54.468721 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22f8b6-9b5f-44c8-81f5-9f3bfe7e4fe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:54.469028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22f8b6-9b5f-44c8-81f5-9f3bfe7e4fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:54.469312 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22f8b6-9b5f-44c8-81f5-9f3bfe7e4fe0 admin admin] PATH_INFO: `/v3/projects/5fdf048be460453d937439116caa2f4f/users/dbfc00a8fc9f4b1f9535619c5a957c81/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:54.471312 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22f8b6-9b5f-44c8-81f5-9f3bfe7e4fe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fdf048be460453d937439116caa2f4f, user_id=dbfc00a8fc9f4b1f9535619c5a957c81, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:53:54.474602 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22f8b6-9b5f-44c8-81f5-9f3bfe7e4fe0 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:53:54.489920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1240/2466] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:53:54 2022] PUT /identity/v3/projects/5fdf048be460453d937439116caa2f4f/users/dbfc00a8fc9f4b1f9535619c5a957c81/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:01.528077 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e54a96-81bf-4c49-934d-4fb2cd6567a2 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:01.528622 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e54a96-81bf-4c49-934d-4fb2cd6567a2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:01.528622 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e54a96-81bf-4c49-934d-4fb2cd6567a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:01.559377 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a3e54a96-81bf-4c49-934d-4fb2cd6567a2 None None] MFA Rules not processed for user `dbfc00a8fc9f4b1f9535619c5a957c81`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:01.560589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a3e54a96-81bf-4c49-934d-4fb2cd6567a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:01.631877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1227/2467] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:54:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:01.643958 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22f8b6-9b5f-44c8-81f5-9f3bfe7e4fe0 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:01.645788 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-2b22f8b6-9b5f-44c8-81f5-9f3bfe7e4fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:01.708255 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22f8b6-9b5f-44c8-81f5-9f3bfe7e4fe0 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:01.711986 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b52b9f-8ca7-4a0f-abac-5a2e2f65aa12 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6105838cbf1409587ee7d9e3be554f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:01.714395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b52b9f-8ca7-4a0f-abac-5a2e2f65aa12 service glance] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:01.714395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b52b9f-8ca7-4a0f-abac-5a2e2f65aa12 service glance] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:01.714395 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b52b9f-8ca7-4a0f-abac-5a2e2f65aa12 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:01.770767 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b52b9f-8ca7-4a0f-abac-5a2e2f65aa12 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:01.772244 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b52b9f-8ca7-4a0f-abac-5a2e2f65aa12 service glance] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:01.781171 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1241/2468] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:02.013845 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89934d14-2139-44c4-a89e-8b73b267cfdf None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:02.014134 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89934d14-2139-44c4-a89e-8b73b267cfdf None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:02.014409 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89934d14-2139-44c4-a89e-8b73b267cfdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:02.034477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-89934d14-2139-44c4-a89e-8b73b267cfdf None None] MFA Rules not processed for user `030614ef56f143f8be6fcc0445ec9568`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:02.035709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-89934d14-2139-44c4-a89e-8b73b267cfdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:02.053631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:02.095151 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1228/2469] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 23 11:54:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:02.229031 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b52b9f-8ca7-4a0f-abac-5a2e2f65aa12 service glance] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:02.230453 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-88b52b9f-8ca7-4a0f-abac-5a2e2f65aa12 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:02.245205 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:02.300847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b52b9f-8ca7-4a0f-abac-5a2e2f65aa12 service glance] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:02.306456 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d614e2-d51e-4e6f-af33-6604bd735ccc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450b1fad96604fb09d0705895c6d3da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:02.307126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d614e2-d51e-4e6f-af33-6604bd735ccc service swift] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:02.307411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d614e2-d51e-4e6f-af33-6604bd735ccc service swift] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:02.307685 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d614e2-d51e-4e6f-af33-6604bd735ccc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:02.316538 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d614e2-d51e-4e6f-af33-6604bd735ccc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:02.318128 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d614e2-d51e-4e6f-af33-6604bd735ccc service swift] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:02.326100 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1242/2470] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 23 11:54:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:54:02.334424 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89934d14-2139-44c4-a89e-8b73b267cfdf None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:02.414967 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89934d14-2139-44c4-a89e-8b73b267cfdf None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:02.418619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85639a16-d848-4eba-801f-700dfce8a2ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:02.419268 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85639a16-d848-4eba-801f-700dfce8a2ed service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:02.419721 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85639a16-d848-4eba-801f-700dfce8a2ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:02.420002 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85639a16-d848-4eba-801f-700dfce8a2ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:02.421802 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-85639a16-d848-4eba-801f-700dfce8a2ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:02.542234 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85639a16-d848-4eba-801f-700dfce8a2ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:02.543802 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85639a16-d848-4eba-801f-700dfce8a2ed service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:02.561475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1229/2471] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:05.222322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d614e2-d51e-4e6f-af33-6604bd735ccc service swift] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:05.235807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d614e2-d51e-4e6f-af33-6604bd735ccc service swift] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:05.245056 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4f33b-2686-4965-bd7b-6eae4bf7701b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:05.246173 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4f33b-2686-4965-bd7b-6eae4bf7701b service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:05.246173 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4f33b-2686-4965-bd7b-6eae4bf7701b service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:05.249143 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4f33b-2686-4965-bd7b-6eae4bf7701b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:05.254439 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-6ab4f33b-2686-4965-bd7b-6eae4bf7701b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:05.278882 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4f33b-2686-4965-bd7b-6eae4bf7701b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:05.280664 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4f33b-2686-4965-bd7b-6eae4bf7701b service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:05.297031 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1243/2472] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:12.552120 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=69 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:12.801433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860b8a4-fe71-4d61-ae7a-fe5f6ba2aadf None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:12.801810 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860b8a4-fe71-4d61-ae7a-fe5f6ba2aadf None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:12.802183 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860b8a4-fe71-4d61-ae7a-fe5f6ba2aadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:12.820180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-2860b8a4-fe71-4d61-ae7a-fe5f6ba2aadf None None] MFA Rules not processed for user `15a4c8e15cd44904805c2519c4b6864a`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:12.821570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2860b8a4-fe71-4d61-ae7a-fe5f6ba2aadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:12.828720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1230/2473] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:54:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:54:14.266614 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4f33b-2686-4965-bd7b-6eae4bf7701b service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:14.275341 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4f33b-2686-4965-bd7b-6eae4bf7701b service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:14.278988 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565c42e-fe0c-4861-83e0-6c05a60b303a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:14.279623 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565c42e-fe0c-4861-83e0-6c05a60b303a service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:14.279908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565c42e-fe0c-4861-83e0-6c05a60b303a service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:14.280185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565c42e-fe0c-4861-83e0-6c05a60b303a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:14.282016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-4565c42e-fe0c-4861-83e0-6c05a60b303a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:14.343686 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565c42e-fe0c-4861-83e0-6c05a60b303a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:14.345113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565c42e-fe0c-4861-83e0-6c05a60b303a service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:14.355052 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1244/2474] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:14 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:17.358027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580ab81-d78f-430c-8d08-31edaf69236c None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:17.358554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580ab81-d78f-430c-8d08-31edaf69236c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:17.358685 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580ab81-d78f-430c-8d08-31edaf69236c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:17.400518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5580ab81-d78f-430c-8d08-31edaf69236c None None] MFA Rules not processed for user `030614ef56f143f8be6fcc0445ec9568`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:17.402006 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5580ab81-d78f-430c-8d08-31edaf69236c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:17.421781 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1231/2475] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 23 11:54:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:17.431144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565c42e-fe0c-4861-83e0-6c05a60b303a service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:17.460712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565c42e-fe0c-4861-83e0-6c05a60b303a service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:17.473726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6799f0a-baa5-473a-8e07-4cf3ef90c415 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450b1fad96604fb09d0705895c6d3da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:17.474615 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6799f0a-baa5-473a-8e07-4cf3ef90c415 service swift] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:17.475048 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6799f0a-baa5-473a-8e07-4cf3ef90c415 service swift] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:17.475471 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6799f0a-baa5-473a-8e07-4cf3ef90c415 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:17.494010 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6799f0a-baa5-473a-8e07-4cf3ef90c415 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:17.496936 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6799f0a-baa5-473a-8e07-4cf3ef90c415 service swift] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:17.520824 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1245/2476] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 23 11:54:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:54:27.387603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=6 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:27.502925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=68 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:35.722830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e49d0-8b57-4cfe-ac3a-968939252ee1 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.723698 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e49d0-8b57-4cfe-ac3a-968939252ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.723980 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e49d0-8b57-4cfe-ac3a-968939252ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.744231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-541e49d0-8b57-4cfe-ac3a-968939252ee1 None None] MFA Rules not processed for user `030614ef56f143f8be6fcc0445ec9568`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:35.745443 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-541e49d0-8b57-4cfe-ac3a-968939252ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:35.752876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1232/2477] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 23 11:54:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:35.766770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6799f0a-baa5-473a-8e07-4cf3ef90c415 service swift] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.776714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6799f0a-baa5-473a-8e07-4cf3ef90c415 service swift] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.781823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa986992-121e-4efa-aded-1588471dc42d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450b1fad96604fb09d0705895c6d3da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.782562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa986992-121e-4efa-aded-1588471dc42d service swift] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.782915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa986992-121e-4efa-aded-1588471dc42d service swift] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.783246 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa986992-121e-4efa-aded-1588471dc42d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.799295 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa986992-121e-4efa-aded-1588471dc42d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.800875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa986992-121e-4efa-aded-1588471dc42d service swift] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.809592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1246/2478] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 23 11:54:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:54:43.221359 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e49d0-8b57-4cfe-ac3a-968939252ee1 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:43.233711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e49d0-8b57-4cfe-ac3a-968939252ee1 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:43.238151 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9fc670-29d1-4f55-954d-24a5c907864a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:43.239374 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fc670-29d1-4f55-954d-24a5c907864a service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:43.239625 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fc670-29d1-4f55-954d-24a5c907864a service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:43.239904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fc670-29d1-4f55-954d-24a5c907864a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:43.242720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3d9fc670-29d1-4f55-954d-24a5c907864a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:43.253133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9fc670-29d1-4f55-954d-24a5c907864a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:43.254654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9fc670-29d1-4f55-954d-24a5c907864a service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:43.273822 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1233/2479] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:45.806705 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=6 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:53.259414 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=9 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:53.427136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e616933-8a70-486f-a698-4b2d2d141306 None None] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:53.427717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e616933-8a70-486f-a698-4b2d2d141306 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:53.427717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e616933-8a70-486f-a698-4b2d2d141306 None None] PATH_INFO: `/` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:53.429162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1247/2480] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 23 11:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:54:53.434880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5fad4-f403-4b10-a22f-080b1d7a924f None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:53.435295 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5fad4-f403-4b10-a22f-080b1d7a924f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:53.435643 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5fad4-f403-4b10-a22f-080b1d7a924f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:53.456577 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a1b5fad4-f403-4b10-a22f-080b1d7a924f None None] MFA Rules not processed for user `161c5ed1e47e450290a1f428ee100b68`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:53.457872 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a1b5fad4-f403-4b10-a22f-080b1d7a924f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:53.467502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1234/2481] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 23 11:54:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:53.480048 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e616933-8a70-486f-a698-4b2d2d141306 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:53.508907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e616933-8a70-486f-a698-4b2d2d141306 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:53.509448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2f260-5a9f-4b36-b3d0-689ec0215a33 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6105838cbf1409587ee7d9e3be554f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:53.510089 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2f260-5a9f-4b36-b3d0-689ec0215a33 service glance] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:53.510375 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2f260-5a9f-4b36-b3d0-689ec0215a33 service glance] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:53.510699 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2f260-5a9f-4b36-b3d0-689ec0215a33 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:53.521080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2f260-5a9f-4b36-b3d0-689ec0215a33 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:53.522506 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2f260-5a9f-4b36-b3d0-689ec0215a33 service glance] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:53.533312 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1248/2482] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:53.901569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5fad4-f403-4b10-a22f-080b1d7a924f None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:53.903076 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a1b5fad4-f403-4b10-a22f-080b1d7a924f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:53.988375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5fad4-f403-4b10-a22f-080b1d7a924f None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:53.995767 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6e40f-c456-43c1-80c7-d53f470dc30f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac04b32777ca4f929e9e59deee6e0260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:53.996838 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6e40f-c456-43c1-80c7-d53f470dc30f service placement] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:54.002225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6e40f-c456-43c1-80c7-d53f470dc30f service placement] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:54.002225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6e40f-c456-43c1-80c7-d53f470dc30f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:54.002225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-69e6e40f-c456-43c1-80c7-d53f470dc30f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:54.016693 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e6e40f-c456-43c1-80c7-d53f470dc30f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:54.019080 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e6e40f-c456-43c1-80c7-d53f470dc30f service placement] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:54.031759 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1235/2483] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:56.403169 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7903f27-3220-41f0-9426-50fa5b61671d None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:56.403668 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7903f27-3220-41f0-9426-50fa5b61671d None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:56.403788 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7903f27-3220-41f0-9426-50fa5b61671d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:56.421770 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-a7903f27-3220-41f0-9426-50fa5b61671d None None] MFA Rules not processed for user `030614ef56f143f8be6fcc0445ec9568`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:56.423203 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a7903f27-3220-41f0-9426-50fa5b61671d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:56.431370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1249/2484] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 23 11:54:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:56.443155 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6e40f-c456-43c1-80c7-d53f470dc30f service placement] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:56.455352 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6e40f-c456-43c1-80c7-d53f470dc30f service placement] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:56.460434 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32159184-1d00-42a0-a1c5-9e699266b402 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450b1fad96604fb09d0705895c6d3da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:56.461594 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32159184-1d00-42a0-a1c5-9e699266b402 service swift] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:56.461875 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32159184-1d00-42a0-a1c5-9e699266b402 service swift] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:56.462146 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32159184-1d00-42a0-a1c5-9e699266b402 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:56.470143 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32159184-1d00-42a0-a1c5-9e699266b402 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:56.471742 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32159184-1d00-42a0-a1c5-9e699266b402 service swift] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:56.480808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1236/2485] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 23 11:54:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:55:06.416619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=9 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:06.476357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=41 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:15.710344 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7903f27-3220-41f0-9426-50fa5b61671d None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:15.720246 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7903f27-3220-41f0-9426-50fa5b61671d None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:15.724057 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd1e7c-6c9e-4de5-9219-84bb74786a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:15.728918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd1e7c-6c9e-4de5-9219-84bb74786a76 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:15.728918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd1e7c-6c9e-4de5-9219-84bb74786a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:15.728918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd1e7c-6c9e-4de5-9219-84bb74786a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:15.735484 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd1e7c-6c9e-4de5-9219-84bb74786a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:15.736996 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd1e7c-6c9e-4de5-9219-84bb74786a76 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:15.746135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1250/2486] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:19.633830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21a850-d3f9-4227-965f-231b518633b2 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:19.637480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21a850-d3f9-4227-965f-231b518633b2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:19.637480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21a850-d3f9-4227-965f-231b518633b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:19.654714 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-bd21a850-d3f9-4227-965f-231b518633b2 None None] MFA Rules not processed for user `030614ef56f143f8be6fcc0445ec9568`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:19.655983 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bd21a850-d3f9-4227-965f-231b518633b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:19.664068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1237/2487] 10.222.0.13 () {58 vars in 967 bytes} [Thu Jun 23 11:55:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:19.690730 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd1e7c-6c9e-4de5-9219-84bb74786a76 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:19.712232 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd1e7c-6c9e-4de5-9219-84bb74786a76 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:19.712232 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50132e73-4b36-4f5a-91c5-51bbc02f67ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450b1fad96604fb09d0705895c6d3da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:19.712756 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50132e73-4b36-4f5a-91c5-51bbc02f67ad service swift] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:19.713035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50132e73-4b36-4f5a-91c5-51bbc02f67ad service swift] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:19.713307 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50132e73-4b36-4f5a-91c5-51bbc02f67ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:19.722891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50132e73-4b36-4f5a-91c5-51bbc02f67ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:19.724638 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50132e73-4b36-4f5a-91c5-51bbc02f67ad service swift] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:19.733992 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1251/2488] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 23 11:55:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:55:29.649199 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:29.730067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=12 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:31.038251 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21a850-d3f9-4227-965f-231b518633b2 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:31.039755 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bd21a850-d3f9-4227-965f-231b518633b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:31.062204 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21a850-d3f9-4227-965f-231b518633b2 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:31.072978 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9b061-9440-439c-9f78-a3a98ababb9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:31.081875 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9b061-9440-439c-9f78-a3a98ababb9b service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:31.082401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9b061-9440-439c-9f78-a3a98ababb9b service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:31.082874 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9b061-9440-439c-9f78-a3a98ababb9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:31.085391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-55c9b061-9440-439c-9f78-a3a98ababb9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:31.107883 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9b061-9440-439c-9f78-a3a98ababb9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:31.110523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9b061-9440-439c-9f78-a3a98ababb9b service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:31.154059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1238/2489] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:33.301064 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50132e73-4b36-4f5a-91c5-51bbc02f67ad service swift] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:33.302618 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-50132e73-4b36-4f5a-91c5-51bbc02f67ad service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:33.360872 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50132e73-4b36-4f5a-91c5-51bbc02f67ad service swift] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:33.364583 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4dfda-61ab-47ed-9758-79ec131ed9c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:33.365164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4dfda-61ab-47ed-9758-79ec131ed9c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:33.365447 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4dfda-61ab-47ed-9758-79ec131ed9c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:33.365718 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4dfda-61ab-47ed-9758-79ec131ed9c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:33.367469 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-57d4dfda-61ab-47ed-9758-79ec131ed9c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:33.460749 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4dfda-61ab-47ed-9758-79ec131ed9c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:33.462229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4dfda-61ab-47ed-9758-79ec131ed9c4 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:33.482869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1252/2490] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:37.405859 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9b061-9440-439c-9f78-a3a98ababb9b service nova] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:37.415488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9b061-9440-439c-9f78-a3a98ababb9b service nova] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:37.419108 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea74795-5b52-49f1-8afb-d958013d116a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:37.419838 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea74795-5b52-49f1-8afb-d958013d116a service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:37.420116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea74795-5b52-49f1-8afb-d958013d116a service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:37.420475 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea74795-5b52-49f1-8afb-d958013d116a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:37.422348 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4ea74795-5b52-49f1-8afb-d958013d116a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:37.492023 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea74795-5b52-49f1-8afb-d958013d116a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:37.492023 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea74795-5b52-49f1-8afb-d958013d116a service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:37.501009 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1239/2491] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:37.978500 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4dfda-61ab-47ed-9758-79ec131ed9c4 service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:37.980241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-57d4dfda-61ab-47ed-9758-79ec131ed9c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:37.992446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4dfda-61ab-47ed-9758-79ec131ed9c4 service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:37.996121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6aa0a-4230-4fca-a867-a26b6a9be1e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:37.996837 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6aa0a-4230-4fca-a867-a26b6a9be1e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:37.997233 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6aa0a-4230-4fca-a867-a26b6a9be1e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:37.997570 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6aa0a-4230-4fca-a867-a26b6a9be1e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:37.999401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-8fe6aa0a-4230-4fca-a867-a26b6a9be1e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:38.012157 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6aa0a-4230-4fca-a867-a26b6a9be1e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:38.013589 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6aa0a-4230-4fca-a867-a26b6a9be1e3 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:38.023393 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1253/2492] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:41.369717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea74795-5b52-49f1-8afb-d958013d116a service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:41.371205 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4ea74795-5b52-49f1-8afb-d958013d116a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:41.397453 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea74795-5b52-49f1-8afb-d958013d116a service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:41.403854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d682075-03ec-43f6-8d2c-2d5e39d0bfb4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac04b32777ca4f929e9e59deee6e0260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:41.404815 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d682075-03ec-43f6-8d2c-2d5e39d0bfb4 service placement] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:41.413149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d682075-03ec-43f6-8d2c-2d5e39d0bfb4 service placement] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:41.413149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d682075-03ec-43f6-8d2c-2d5e39d0bfb4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:41.413149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-8d682075-03ec-43f6-8d2c-2d5e39d0bfb4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:41.423477 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d682075-03ec-43f6-8d2c-2d5e39d0bfb4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:41.426676 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d682075-03ec-43f6-8d2c-2d5e39d0bfb4 service placement] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:41.439203 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1240/2493] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:41.522377 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6aa0a-4230-4fca-a867-a26b6a9be1e3 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:41.522377 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-8fe6aa0a-4230-4fca-a867-a26b6a9be1e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:41.535759 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6aa0a-4230-4fca-a867-a26b6a9be1e3 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:41.540385 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc5adb-90d4-4fa8-b770-2ab89068aad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:41.541253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc5adb-90d4-4fa8-b770-2ab89068aad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:41.541530 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc5adb-90d4-4fa8-b770-2ab89068aad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:41.541805 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc5adb-90d4-4fa8-b770-2ab89068aad6 admin admin] PATH_INFO: `/v3/users/d1b988f3241a46308b7c3683e2943b78` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:41.543055 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc5adb-90d4-4fa8-b770-2ab89068aad6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1b988f3241a46308b7c3683e2943b78)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:41.545199 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc5adb-90d4-4fa8-b770-2ab89068aad6 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:41.813203 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f5bc5adb-90d4-4fa8-b770-2ab89068aad6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1b988f3241a46308b7c3683e2943b78'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:41.824708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f5bc5adb-90d4-4fa8-b770-2ab89068aad6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1b988f3241a46308b7c3683e2943b78'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:41.825728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f5bc5adb-90d4-4fa8-b770-2ab89068aad6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1b988f3241a46308b7c3683e2943b78'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:41.826491 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f5bc5adb-90d4-4fa8-b770-2ab89068aad6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1b988f3241a46308b7c3683e2943b78'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:41.835958 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:41.840505 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f5bc5adb-90d4-4fa8-b770-2ab89068aad6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1b988f3241a46308b7c3683e2943b78'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:41.962343 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1254/2494] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:41 2022] DELETE /identity/v3/users/d1b988f3241a46308b7c3683e2943b78 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:42.107820 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d682075-03ec-43f6-8d2c-2d5e39d0bfb4 service placement] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:42.109010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-8d682075-03ec-43f6-8d2c-2d5e39d0bfb4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:42.130243 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d682075-03ec-43f6-8d2c-2d5e39d0bfb4 service placement] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:42.186161 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1676e5b-21d3-4e75-8d51-a4c0c09698de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:42.187069 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1676e5b-21d3-4e75-8d51-a4c0c09698de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:42.187977 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1676e5b-21d3-4e75-8d51-a4c0c09698de admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:42.188376 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1676e5b-21d3-4e75-8d51-a4c0c09698de admin admin] PATH_INFO: `/v3/users/b7fd8323d29d445f903e2103453e8ae6` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:42.202150 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1676e5b-21d3-4e75-8d51-a4c0c09698de admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7fd8323d29d445f903e2103453e8ae6)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:42.204173 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1676e5b-21d3-4e75-8d51-a4c0c09698de admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:42.226863 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc5adb-90d4-4fa8-b770-2ab89068aad6 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:42.232339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f5bc5adb-90d4-4fa8-b770-2ab89068aad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:42.243605 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc5adb-90d4-4fa8-b770-2ab89068aad6 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:42.248235 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826416c4-6362-41c5-bad9-1f399450fbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:42.249128 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826416c4-6362-41c5-bad9-1f399450fbbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:42.249416 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826416c4-6362-41c5-bad9-1f399450fbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:42.249761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826416c4-6362-41c5-bad9-1f399450fbbb admin admin] PATH_INFO: `/v3/projects/45db495ff8c04b55b44c80a3ab412fb9` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:42.251647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826416c4-6362-41c5-bad9-1f399450fbbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=45db495ff8c04b55b44c80a3ab412fb9)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:42.253659 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826416c4-6362-41c5-bad9-1f399450fbbb admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:42.455446 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f1676e5b-21d3-4e75-8d51-a4c0c09698de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7fd8323d29d445f903e2103453e8ae6'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:42.456030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-826416c4-6362-41c5-bad9-1f399450fbbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45db495ff8c04b55b44c80a3ab412fb9', 'request_id': 'req-826416c4-6362-41c5-bad9-1f399450fbbb'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:42.462306 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f1676e5b-21d3-4e75-8d51-a4c0c09698de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7fd8323d29d445f903e2103453e8ae6'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:42.463014 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f1676e5b-21d3-4e75-8d51-a4c0c09698de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7fd8323d29d445f903e2103453e8ae6'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:42.463366 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-826416c4-6362-41c5-bad9-1f399450fbbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45db495ff8c04b55b44c80a3ab412fb9', 'request_id': 'req-826416c4-6362-41c5-bad9-1f399450fbbb'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:42.463709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f1676e5b-21d3-4e75-8d51-a4c0c09698de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7fd8323d29d445f903e2103453e8ae6'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:42.465608 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:42.476515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f1676e5b-21d3-4e75-8d51-a4c0c09698de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7fd8323d29d445f903e2103453e8ae6'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:42.573569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-826416c4-6362-41c5-bad9-1f399450fbbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45db495ff8c04b55b44c80a3ab412fb9', 'request_id': 'req-826416c4-6362-41c5-bad9-1f399450fbbb'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:42.584790 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-826416c4-6362-41c5-bad9-1f399450fbbb admin admin] The token cache is being invalidate because project 45db495ff8c04b55b44c80a3ab412fb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:42.585081 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-826416c4-6362-41c5-bad9-1f399450fbbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:42.585702 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-826416c4-6362-41c5-bad9-1f399450fbbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:42.587298 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1255/2495] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:42 2022] DELETE /identity/v3/projects/45db495ff8c04b55b44c80a3ab412fb9 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:42.592230 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcf437-cd56-42d9-b7ab-4124552e0607 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:42.592544 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcf437-cd56-42d9-b7ab-4124552e0607 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:42.592823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcf437-cd56-42d9-b7ab-4124552e0607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:42.611458 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-3cbcf437-cd56-42d9-b7ab-4124552e0607 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:42.612641 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-3cbcf437-cd56-42d9-b7ab-4124552e0607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:42.670234 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1256/2496] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:42.676941 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1241/2497] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:42 2022] DELETE /identity/v3/users/b7fd8323d29d445f903e2103453e8ae6 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:42.678803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e06dac-c5fc-4a12-8fd6-dd0612f9689b None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:42.679095 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e06dac-c5fc-4a12-8fd6-dd0612f9689b None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:42.679378 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e06dac-c5fc-4a12-8fd6-dd0612f9689b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:42.697690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-56e06dac-c5fc-4a12-8fd6-dd0612f9689b None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:42.698976 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-56e06dac-c5fc-4a12-8fd6-dd0612f9689b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:42.759783 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1257/2498] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:42.763827 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1676e5b-21d3-4e75-8d51-a4c0c09698de admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:42.765241 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f1676e5b-21d3-4e75-8d51-a4c0c09698de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:42.779327 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1676e5b-21d3-4e75-8d51-a4c0c09698de admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:42.784896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad565fc7-a9f5-4741-910f-44fb0d34fbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:42.785553 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad565fc7-a9f5-4741-910f-44fb0d34fbb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:42.785832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad565fc7-a9f5-4741-910f-44fb0d34fbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:42.786099 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad565fc7-a9f5-4741-910f-44fb0d34fbb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:42.786847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad565fc7-a9f5-4741-910f-44fb0d34fbb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:55:42.787252 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad565fc7-a9f5-4741-910f-44fb0d34fbb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:42.788988 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad565fc7-a9f5-4741-910f-44fb0d34fbb9 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:42.801083 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1242/2499] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:55:42.805508 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e06dac-c5fc-4a12-8fd6-dd0612f9689b None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:42.815993 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e06dac-c5fc-4a12-8fd6-dd0612f9689b None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:42.820955 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb848446-ced2-4eea-b4af-200f21906163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:42.821609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb848446-ced2-4eea-b4af-200f21906163 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:42.821896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb848446-ced2-4eea-b4af-200f21906163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:42.822174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb848446-ced2-4eea-b4af-200f21906163 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:42.823005 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb848446-ced2-4eea-b4af-200f21906163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:42.824987 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb848446-ced2-4eea-b4af-200f21906163 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:42.830020 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:42.946909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad565fc7-a9f5-4741-910f-44fb0d34fbb9 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:42.951753 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1258/2500] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:42 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:42.951893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ad565fc7-a9f5-4741-910f-44fb0d34fbb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:42.952553 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb848446-ced2-4eea-b4af-200f21906163 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:42.964050 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad565fc7-a9f5-4741-910f-44fb0d34fbb9 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.028474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb848446-ced2-4eea-b4af-200f21906163 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.033810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423530f9-1529-4a1b-9884-2aeedb55fb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.039671 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423530f9-1529-4a1b-9884-2aeedb55fb8f admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.039671 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423530f9-1529-4a1b-9884-2aeedb55fb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.039671 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423530f9-1529-4a1b-9884-2aeedb55fb8f admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.039671 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423530f9-1529-4a1b-9884-2aeedb55fb8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.039671 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423530f9-1529-4a1b-9884-2aeedb55fb8f admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.042944 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40537bc0-d414-4c23-9495-d279abe88523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.043808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40537bc0-d414-4c23-9495-d279abe88523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.044098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40537bc0-d414-4c23-9495-d279abe88523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.044409 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40537bc0-d414-4c23-9495-d279abe88523 admin admin] PATH_INFO: `/v3/projects/ee135767dfc846469d64b658372994a8` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.055786 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40537bc0-d414-4c23-9495-d279abe88523 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee135767dfc846469d64b658372994a8)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.057769 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40537bc0-d414-4c23-9495-d279abe88523 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.116029 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1259/2501] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:42 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:55:43.120214 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423530f9-1529-4a1b-9884-2aeedb55fb8f admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.130181 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423530f9-1529-4a1b-9884-2aeedb55fb8f admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.134827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6d423-9b87-4738-98bf-c808f334ccd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.135464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6d423-9b87-4738-98bf-c808f334ccd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.135754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6d423-9b87-4738-98bf-c808f334ccd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.136030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6d423-9b87-4738-98bf-c808f334ccd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.136768 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6d423-9b87-4738-98bf-c808f334ccd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.138355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6d423-9b87-4738-98bf-c808f334ccd5 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.148860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1260/2502] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:55:43.152574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6d423-9b87-4738-98bf-c808f334ccd5 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.162005 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6d423-9b87-4738-98bf-c808f334ccd5 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.166684 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a61c15-3103-43b7-8925-ceed80df8ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.167684 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a61c15-3103-43b7-8925-ceed80df8ad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.167967 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a61c15-3103-43b7-8925-ceed80df8ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.168237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a61c15-3103-43b7-8925-ceed80df8ad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.169000 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a61c15-3103-43b7-8925-ceed80df8ad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.170726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a61c15-3103-43b7-8925-ceed80df8ad3 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.183918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1261/2503] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:55:43.184075 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-40537bc0-d414-4c23-9495-d279abe88523 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee135767dfc846469d64b658372994a8', 'request_id': 'req-40537bc0-d414-4c23-9495-d279abe88523'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:43.185236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a61c15-3103-43b7-8925-ceed80df8ad3 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.189283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-40537bc0-d414-4c23-9495-d279abe88523 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee135767dfc846469d64b658372994a8', 'request_id': 'req-40537bc0-d414-4c23-9495-d279abe88523'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:43.196289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a61c15-3103-43b7-8925-ceed80df8ad3 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.253593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b4b40-b698-437a-bb45-769717ff6a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.254460 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b4b40-b698-437a-bb45-769717ff6a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.254776 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b4b40-b698-437a-bb45-769717ff6a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.255051 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b4b40-b698-437a-bb45-769717ff6a0b admin admin] PATH_INFO: `/v3/projects/fd8e176b4c9b4898ae5fc3b201e86e04/users/87d269c842ac4f01bfa19335784a7c57/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.258777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-40537bc0-d414-4c23-9495-d279abe88523 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee135767dfc846469d64b658372994a8', 'request_id': 'req-40537bc0-d414-4c23-9495-d279abe88523'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:43.269993 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b4b40-b698-437a-bb45-769717ff6a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd8e176b4c9b4898ae5fc3b201e86e04, user_id=87d269c842ac4f01bfa19335784a7c57, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.270832 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-40537bc0-d414-4c23-9495-d279abe88523 admin admin] The token cache is being invalidate because project ee135767dfc846469d64b658372994a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:43.271149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-40537bc0-d414-4c23-9495-d279abe88523 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:43.272030 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-40537bc0-d414-4c23-9495-d279abe88523 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:43.272800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b4b40-b698-437a-bb45-769717ff6a0b admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.273815 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1243/2504] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:42 2022] DELETE /identity/v3/projects/ee135767dfc846469d64b658372994a8 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:43.278525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a52cf-490f-41f5-b873-39bfe2d80f34 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.280210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a52cf-490f-41f5-b873-39bfe2d80f34 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.280210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a52cf-490f-41f5-b873-39bfe2d80f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.296785 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-357a52cf-490f-41f5-b873-39bfe2d80f34 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:43.298012 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-357a52cf-490f-41f5-b873-39bfe2d80f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:43.305971 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1244/2505] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:43.313381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46359ae0-6744-4414-9773-241aef7c641d None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.314495 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46359ae0-6744-4414-9773-241aef7c641d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.314495 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46359ae0-6744-4414-9773-241aef7c641d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.332798 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-46359ae0-6744-4414-9773-241aef7c641d None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:43.333968 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-46359ae0-6744-4414-9773-241aef7c641d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:43.341710 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1245/2506] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:43.345658 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46359ae0-6744-4414-9773-241aef7c641d None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.354483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46359ae0-6744-4414-9773-241aef7c641d None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.359198 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4f7a9-c3e6-4abb-9c55-812dc7828639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.359808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4f7a9-c3e6-4abb-9c55-812dc7828639 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.360093 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4f7a9-c3e6-4abb-9c55-812dc7828639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.360623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4f7a9-c3e6-4abb-9c55-812dc7828639 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.361870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd4f7a9-c3e6-4abb-9c55-812dc7828639 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:55:43.361870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd4f7a9-c3e6-4abb-9c55-812dc7828639 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.363163 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd4f7a9-c3e6-4abb-9c55-812dc7828639 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.368613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1262/2507] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:43 2022] PUT /identity/v3/projects/fd8e176b4c9b4898ae5fc3b201e86e04/users/87d269c842ac4f01bfa19335784a7c57/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:43.380285 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1246/2508] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:55:43.380437 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b4b40-b698-437a-bb45-769717ff6a0b admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.381522 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc42cee-ef17-4ece-85fb-8ee9915bcadd None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.381812 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc42cee-ef17-4ece-85fb-8ee9915bcadd None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.382086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc42cee-ef17-4ece-85fb-8ee9915bcadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.407991 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-7cc42cee-ef17-4ece-85fb-8ee9915bcadd None None] MFA Rules not processed for user `87d269c842ac4f01bfa19335784a7c57`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:43.409195 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7cc42cee-ef17-4ece-85fb-8ee9915bcadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:43.414297 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:43.423362 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:43.444325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b4b40-b698-437a-bb45-769717ff6a0b admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.449236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d8f73-f2dd-4454-af04-5f9c43c35f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.449896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d8f73-f2dd-4454-af04-5f9c43c35f82 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.450185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d8f73-f2dd-4454-af04-5f9c43c35f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.450461 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d8f73-f2dd-4454-af04-5f9c43c35f82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.451190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d8f73-f2dd-4454-af04-5f9c43c35f82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.453308 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d8f73-f2dd-4454-af04-5f9c43c35f82 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.478847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1247/2509] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:55:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:43.492554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc42cee-ef17-4ece-85fb-8ee9915bcadd None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.493151 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7cc42cee-ef17-4ece-85fb-8ee9915bcadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:43.558025 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1263/2510] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:43 2022] POST /identity/v3/projects => generated 371 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:43.562260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d8f73-f2dd-4454-af04-5f9c43c35f82 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.569351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc42cee-ef17-4ece-85fb-8ee9915bcadd None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.627665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d8f73-f2dd-4454-af04-5f9c43c35f82 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.631841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf18848-c2d1-456d-b9d4-14cc252056f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.632376 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17f3da-d80f-4c77-a31b-647c2a798a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.632507 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf18848-c2d1-456d-b9d4-14cc252056f6 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.632700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf18848-c2d1-456d-b9d4-14cc252056f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.632983 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf18848-c2d1-456d-b9d4-14cc252056f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.633080 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17f3da-d80f-4c77-a31b-647c2a798a72 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.633309 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17f3da-d80f-4c77-a31b-647c2a798a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.633586 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17f3da-d80f-4c77-a31b-647c2a798a72 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.634275 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17f3da-d80f-4c77-a31b-647c2a798a72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.636316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17f3da-d80f-4c77-a31b-647c2a798a72 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.677063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:43.691620 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf18848-c2d1-456d-b9d4-14cc252056f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.693373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf18848-c2d1-456d-b9d4-14cc252056f6 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.702330 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1248/2511] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:55:43.715756 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1264/2512] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:43 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:55:43.719690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf18848-c2d1-456d-b9d4-14cc252056f6 service nova] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.730733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf18848-c2d1-456d-b9d4-14cc252056f6 service nova] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.738550 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0f854-e677-4af8-bde0-578c7575fa23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.738741 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0f854-e677-4af8-bde0-578c7575fa23 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.739029 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0f854-e677-4af8-bde0-578c7575fa23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.739306 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0f854-e677-4af8-bde0-578c7575fa23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.740066 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0f854-e677-4af8-bde0-578c7575fa23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.741694 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0f854-e677-4af8-bde0-578c7575fa23 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.761513 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1249/2513] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:55:43.765388 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17f3da-d80f-4c77-a31b-647c2a798a72 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.778017 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17f3da-d80f-4c77-a31b-647c2a798a72 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.782489 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d4d8e-22a1-42b8-b9f9-e3d6dc213feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.783634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d4d8e-22a1-42b8-b9f9-e3d6dc213feb admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.783984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d4d8e-22a1-42b8-b9f9-e3d6dc213feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.784328 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d4d8e-22a1-42b8-b9f9-e3d6dc213feb admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.785157 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d4d8e-22a1-42b8-b9f9-e3d6dc213feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.786932 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d4d8e-22a1-42b8-b9f9-e3d6dc213feb admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.798482 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1265/2514] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:43.803569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0f854-e677-4af8-bde0-578c7575fa23 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.814752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0f854-e677-4af8-bde0-578c7575fa23 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.819669 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51640b95-dc45-4178-b3cc-4310b5cdadb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.820578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51640b95-dc45-4178-b3cc-4310b5cdadb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.820862 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51640b95-dc45-4178-b3cc-4310b5cdadb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.822619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51640b95-dc45-4178-b3cc-4310b5cdadb2 admin admin] PATH_INFO: `/v3/projects/6dab0670bac3443abac22ad78e103076/users/c2da36c53e064fedb65ab202d265eb12/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.840231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51640b95-dc45-4178-b3cc-4310b5cdadb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dab0670bac3443abac22ad78e103076, user_id=c2da36c53e064fedb65ab202d265eb12, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.840445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51640b95-dc45-4178-b3cc-4310b5cdadb2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.916629 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1250/2515] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:43 2022] PUT /identity/v3/projects/6dab0670bac3443abac22ad78e103076/users/c2da36c53e064fedb65ab202d265eb12/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:43.927421 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2774d21-184c-4cd2-90e8-bff34b7a2db4 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.927638 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2774d21-184c-4cd2-90e8-bff34b7a2db4 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.927930 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2774d21-184c-4cd2-90e8-bff34b7a2db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.945580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51640b95-dc45-4178-b3cc-4310b5cdadb2 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.947993 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-51640b95-dc45-4178-b3cc-4310b5cdadb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:43.962271 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-d2774d21-184c-4cd2-90e8-bff34b7a2db4 None None] MFA Rules not processed for user `c2da36c53e064fedb65ab202d265eb12`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:43.963513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d2774d21-184c-4cd2-90e8-bff34b7a2db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:44.015221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51640b95-dc45-4178-b3cc-4310b5cdadb2 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:44.020430 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222e728-64ff-4748-bb3e-c1e7a2be117a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:44.021312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222e728-64ff-4748-bb3e-c1e7a2be117a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:44.021599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222e728-64ff-4748-bb3e-c1e7a2be117a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:44.021994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222e728-64ff-4748-bb3e-c1e7a2be117a admin admin] PATH_INFO: `/v3/users/87d269c842ac4f01bfa19335784a7c57` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:44.023439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222e728-64ff-4748-bb3e-c1e7a2be117a admin admin] RBAC: Authorizing `identity:delete_user(user_id=87d269c842ac4f01bfa19335784a7c57)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:44.025603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222e728-64ff-4748-bb3e-c1e7a2be117a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:44.029819 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1266/2516] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:55:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:44.034634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2774d21-184c-4cd2-90e8-bff34b7a2db4 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:44.045057 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2774d21-184c-4cd2-90e8-bff34b7a2db4 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:44.050451 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873c0e2-f825-4695-9a5d-30122242fdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:44.051139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873c0e2-f825-4695-9a5d-30122242fdbb admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:44.051432 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873c0e2-f825-4695-9a5d-30122242fdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:44.051709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873c0e2-f825-4695-9a5d-30122242fdbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:44.052409 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873c0e2-f825-4695-9a5d-30122242fdbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:44.054334 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873c0e2-f825-4695-9a5d-30122242fdbb admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:44.158369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1267/2517] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:44 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:55:44.162335 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873c0e2-f825-4695-9a5d-30122242fdbb admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:44.171890 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:44.221864 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9222e728-64ff-4748-bb3e-c1e7a2be117a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87d269c842ac4f01bfa19335784a7c57'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:44.224114 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873c0e2-f825-4695-9a5d-30122242fdbb admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:44.229056 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31eb83-9af3-4282-b3ad-d2cf0b6a7c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:44.229498 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9222e728-64ff-4748-bb3e-c1e7a2be117a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87d269c842ac4f01bfa19335784a7c57'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:44.230031 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9222e728-64ff-4748-bb3e-c1e7a2be117a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87d269c842ac4f01bfa19335784a7c57'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:44.230137 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31eb83-9af3-4282-b3ad-d2cf0b6a7c4b admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:44.230364 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31eb83-9af3-4282-b3ad-d2cf0b6a7c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:44.230682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9222e728-64ff-4748-bb3e-c1e7a2be117a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87d269c842ac4f01bfa19335784a7c57'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:44.230809 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31eb83-9af3-4282-b3ad-d2cf0b6a7c4b admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:44.231397 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31eb83-9af3-4282-b3ad-d2cf0b6a7c4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:44.233346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31eb83-9af3-4282-b3ad-d2cf0b6a7c4b admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:44.243558 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9222e728-64ff-4748-bb3e-c1e7a2be117a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87d269c842ac4f01bfa19335784a7c57'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:44.327508 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1268/2518] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:44 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:55:44.332064 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31eb83-9af3-4282-b3ad-d2cf0b6a7c4b admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:44.333609 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-6a31eb83-9af3-4282-b3ad-d2cf0b6a7c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:44.345670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31eb83-9af3-4282-b3ad-d2cf0b6a7c4b admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:44.350234 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77350c80-3f06-4c90-b3da-ba412786271d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:44.354101 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77350c80-3f06-4c90-b3da-ba412786271d admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:44.354101 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77350c80-3f06-4c90-b3da-ba412786271d admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:44.354101 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77350c80-3f06-4c90-b3da-ba412786271d admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:44.354101 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77350c80-3f06-4c90-b3da-ba412786271d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:44.354101 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77350c80-3f06-4c90-b3da-ba412786271d admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:44.363639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1269/2519] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:55:44.367372 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77350c80-3f06-4c90-b3da-ba412786271d admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:44.377168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77350c80-3f06-4c90-b3da-ba412786271d admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:44.383000 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff9af1-f15b-42aa-95a6-09c679aee52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:44.383891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff9af1-f15b-42aa-95a6-09c679aee52d admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:44.384179 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff9af1-f15b-42aa-95a6-09c679aee52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:44.384457 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff9af1-f15b-42aa-95a6-09c679aee52d admin admin] PATH_INFO: `/v3/projects/8e12a14669a84ee4a76bddfb3fb84ec2/users/640f83e986cd4eb998349f50535ef7ce/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:44.384773 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1251/2520] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:43 2022] DELETE /identity/v3/users/87d269c842ac4f01bfa19335784a7c57 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:44.394586 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17408fd-b90f-44ff-8681-2d1f33bebca5 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:44.394883 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17408fd-b90f-44ff-8681-2d1f33bebca5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:44.395164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17408fd-b90f-44ff-8681-2d1f33bebca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:44.400149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff9af1-f15b-42aa-95a6-09c679aee52d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e12a14669a84ee4a76bddfb3fb84ec2, user_id=640f83e986cd4eb998349f50535ef7ce, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:44.402863 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff9af1-f15b-42aa-95a6-09c679aee52d admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:44.413427 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b17408fd-b90f-44ff-8681-2d1f33bebca5 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:44.414631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b17408fd-b90f-44ff-8681-2d1f33bebca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:44.435345 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:44.476117 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1252/2521] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:55:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:44.486217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17408fd-b90f-44ff-8681-2d1f33bebca5 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:44.488123 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b17408fd-b90f-44ff-8681-2d1f33bebca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:44.517581 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1270/2522] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:44 2022] PUT /identity/v3/projects/8e12a14669a84ee4a76bddfb3fb84ec2/users/640f83e986cd4eb998349f50535ef7ce/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:44.521573 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff9af1-f15b-42aa-95a6-09c679aee52d admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:44.548876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17408fd-b90f-44ff-8681-2d1f33bebca5 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:44.584964 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff9af1-f15b-42aa-95a6-09c679aee52d admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:44.589696 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8ce11-776e-4c7b-a531-7c6e0673b297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:44.590357 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8ce11-776e-4c7b-a531-7c6e0673b297 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:44.590671 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8ce11-776e-4c7b-a531-7c6e0673b297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:44.590943 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8ce11-776e-4c7b-a531-7c6e0673b297 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:44.591661 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd8ce11-776e-4c7b-a531-7c6e0673b297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:44.593249 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd8ce11-776e-4c7b-a531-7c6e0673b297 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:44.598434 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca196be-f528-4644-a910-faed60f03119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:44.599122 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca196be-f528-4644-a910-faed60f03119 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:44.599410 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca196be-f528-4644-a910-faed60f03119 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:44.599685 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca196be-f528-4644-a910-faed60f03119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:44.603847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1271/2523] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:44.608342 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8ce11-776e-4c7b-a531-7c6e0673b297 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:44.609416 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca196be-f528-4644-a910-faed60f03119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:44.610899 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca196be-f528-4644-a910-faed60f03119 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:44.618029 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8ce11-776e-4c7b-a531-7c6e0673b297 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:44.620411 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1253/2524] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:55:44.622611 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6821ef-e3a6-460b-b094-36cce3f89e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:44.623490 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6821ef-e3a6-460b-b094-36cce3f89e72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:44.623942 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6821ef-e3a6-460b-b094-36cce3f89e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:44.624215 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6821ef-e3a6-460b-b094-36cce3f89e72 admin admin] PATH_INFO: `/v3/projects/8e12a14669a84ee4a76bddfb3fb84ec2/users/640f83e986cd4eb998349f50535ef7ce/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:44.626930 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6821ef-e3a6-460b-b094-36cce3f89e72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e12a14669a84ee4a76bddfb3fb84ec2, user_id=640f83e986cd4eb998349f50535ef7ce, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:44.629546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6821ef-e3a6-460b-b094-36cce3f89e72 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:44.640094 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1272/2525] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:44 2022] PUT /identity/v3/projects/8e12a14669a84ee4a76bddfb3fb84ec2/users/640f83e986cd4eb998349f50535ef7ce/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:44.657178 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20187a1-7130-42a5-b481-042aa2cfe65b None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:44.657510 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20187a1-7130-42a5-b481-042aa2cfe65b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:44.657791 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20187a1-7130-42a5-b481-042aa2cfe65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:44.685371 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c20187a1-7130-42a5-b481-042aa2cfe65b None None] MFA Rules not processed for user `640f83e986cd4eb998349f50535ef7ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:44.686689 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c20187a1-7130-42a5-b481-042aa2cfe65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:44.757526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1254/2526] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:55:44 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:44.769711 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6821ef-e3a6-460b-b094-36cce3f89e72 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:44.771514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-bf6821ef-e3a6-460b-b094-36cce3f89e72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:44.825482 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:44.832122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6821ef-e3a6-460b-b094-36cce3f89e72 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:44.835911 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42a08a-c3e6-49d1-9e9a-b01cae7aed39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:44.836565 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42a08a-c3e6-49d1-9e9a-b01cae7aed39 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:44.836851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42a08a-c3e6-49d1-9e9a-b01cae7aed39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:44.837130 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42a08a-c3e6-49d1-9e9a-b01cae7aed39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:44.847291 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42a08a-c3e6-49d1-9e9a-b01cae7aed39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:44.854782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42a08a-c3e6-49d1-9e9a-b01cae7aed39 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:44.859259 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1273/2527] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:44 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:45.072984 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20187a1-7130-42a5-b481-042aa2cfe65b None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:45.074526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c20187a1-7130-42a5-b481-042aa2cfe65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:45.088448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:45.119538 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42a08a-c3e6-49d1-9e9a-b01cae7aed39 service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:45.128705 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42a08a-c3e6-49d1-9e9a-b01cae7aed39 service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:45.132324 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbd87e-57e0-448b-9727-faf1ccc78e84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:45.133046 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbd87e-57e0-448b-9727-faf1ccc78e84 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:45.133297 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbd87e-57e0-448b-9727-faf1ccc78e84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:45.140128 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbd87e-57e0-448b-9727-faf1ccc78e84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:45.140128 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e5bbd87e-57e0-448b-9727-faf1ccc78e84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:45.142213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20187a1-7130-42a5-b481-042aa2cfe65b None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:45.147251 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67457e67-ce0c-4c92-bfe3-9851013840b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:45.148146 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67457e67-ce0c-4c92-bfe3-9851013840b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:45.148437 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67457e67-ce0c-4c92-bfe3-9851013840b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:45.148728 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67457e67-ce0c-4c92-bfe3-9851013840b0 admin admin] PATH_INFO: `/v3/projects/fd8e176b4c9b4898ae5fc3b201e86e04` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:45.149928 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67457e67-ce0c-4c92-bfe3-9851013840b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd8e176b4c9b4898ae5fc3b201e86e04)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:45.151870 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67457e67-ce0c-4c92-bfe3-9851013840b0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:45.194256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bbd87e-57e0-448b-9727-faf1ccc78e84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:45.195743 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bbd87e-57e0-448b-9727-faf1ccc78e84 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:45.204536 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1274/2528] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:55:45.293708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-67457e67-ce0c-4c92-bfe3-9851013840b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd8e176b4c9b4898ae5fc3b201e86e04', 'request_id': 'req-67457e67-ce0c-4c92-bfe3-9851013840b0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:45.300446 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-67457e67-ce0c-4c92-bfe3-9851013840b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd8e176b4c9b4898ae5fc3b201e86e04', 'request_id': 'req-67457e67-ce0c-4c92-bfe3-9851013840b0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:45.378642 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-67457e67-ce0c-4c92-bfe3-9851013840b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd8e176b4c9b4898ae5fc3b201e86e04', 'request_id': 'req-67457e67-ce0c-4c92-bfe3-9851013840b0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:45.389624 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-67457e67-ce0c-4c92-bfe3-9851013840b0 admin admin] The token cache is being invalidate because project fd8e176b4c9b4898ae5fc3b201e86e04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:45.389875 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-67457e67-ce0c-4c92-bfe3-9851013840b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:45.390542 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-67457e67-ce0c-4c92-bfe3-9851013840b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:45.392248 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1255/2529] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:45 2022] DELETE /identity/v3/projects/fd8e176b4c9b4898ae5fc3b201e86e04 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:45.397429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eaf5f-0a8d-44f3-b051-3f8994f7fe67 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:45.397813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eaf5f-0a8d-44f3-b051-3f8994f7fe67 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:45.398164 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eaf5f-0a8d-44f3-b051-3f8994f7fe67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:45.417744 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-081eaf5f-0a8d-44f3-b051-3f8994f7fe67 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:45.418963 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-081eaf5f-0a8d-44f3-b051-3f8994f7fe67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:45.483103 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1275/2530] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:45.491812 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c8957-7752-4b5e-892d-19aec8b40373 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:45.492225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c8957-7752-4b5e-892d-19aec8b40373 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:45.492371 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c8957-7752-4b5e-892d-19aec8b40373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:45.511831 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-632c8957-7752-4b5e-892d-19aec8b40373 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:45.518806 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-632c8957-7752-4b5e-892d-19aec8b40373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:45.522772 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1256/2531] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:45.527069 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081eaf5f-0a8d-44f3-b051-3f8994f7fe67 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:45.537463 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081eaf5f-0a8d-44f3-b051-3f8994f7fe67 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:45.542068 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f2e8a-4d39-4580-8f7d-1da9a5130bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:45.542794 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f2e8a-4d39-4580-8f7d-1da9a5130bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:45.543093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f2e8a-4d39-4580-8f7d-1da9a5130bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:45.543373 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f2e8a-4d39-4580-8f7d-1da9a5130bb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:45.544190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f2e8a-4d39-4580-8f7d-1da9a5130bb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:55:45.544459 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f2e8a-4d39-4580-8f7d-1da9a5130bb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:45.546124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f2e8a-4d39-4580-8f7d-1da9a5130bb6 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:45.558692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1276/2532] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:55:45.563055 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c8957-7752-4b5e-892d-19aec8b40373 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:45.572478 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c8957-7752-4b5e-892d-19aec8b40373 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:45.577177 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb34425-0825-4249-ac63-9fcf860b0d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:45.577820 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb34425-0825-4249-ac63-9fcf860b0d8e admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:45.581891 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb34425-0825-4249-ac63-9fcf860b0d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:45.581891 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb34425-0825-4249-ac63-9fcf860b0d8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:45.581891 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb34425-0825-4249-ac63-9fcf860b0d8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:45.581891 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb34425-0825-4249-ac63-9fcf860b0d8e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:45.678848 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1257/2533] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:45 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:45.682930 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f2e8a-4d39-4580-8f7d-1da9a5130bb6 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:45.692989 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f2e8a-4d39-4580-8f7d-1da9a5130bb6 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:45.749687 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2c114-79ff-43fd-9096-fd3211c08624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:45.750341 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2c114-79ff-43fd-9096-fd3211c08624 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:45.750672 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2c114-79ff-43fd-9096-fd3211c08624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:45.750946 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2c114-79ff-43fd-9096-fd3211c08624 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:45.752088 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2c114-79ff-43fd-9096-fd3211c08624 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:45.754004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2c114-79ff-43fd-9096-fd3211c08624 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:45.911480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1277/2534] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:45 2022] POST /identity/v3/users => generated 286 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:45.915532 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb34425-0825-4249-ac63-9fcf860b0d8e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:45.925770 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb34425-0825-4249-ac63-9fcf860b0d8e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:45.930268 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279764e-0c11-4442-a322-959cac4a8ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:45.930964 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279764e-0c11-4442-a322-959cac4a8ccf admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:45.931249 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279764e-0c11-4442-a322-959cac4a8ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:45.931530 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279764e-0c11-4442-a322-959cac4a8ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:45.932261 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d279764e-0c11-4442-a322-959cac4a8ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:45.933864 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d279764e-0c11-4442-a322-959cac4a8ccf admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:45.944539 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1258/2535] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:45.948387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2c114-79ff-43fd-9096-fd3211c08624 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:45.955377 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:45.959356 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2c114-79ff-43fd-9096-fd3211c08624 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:45.963968 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a7987-ced4-4d4a-982e-559fac0c4043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:45.964618 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a7987-ced4-4d4a-982e-559fac0c4043 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:45.964899 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a7987-ced4-4d4a-982e-559fac0c4043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:45.965171 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a7987-ced4-4d4a-982e-559fac0c4043 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:45.965905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a7987-ced4-4d4a-982e-559fac0c4043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:45.967528 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a7987-ced4-4d4a-982e-559fac0c4043 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:45.978626 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1278/2536] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:45.982901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279764e-0c11-4442-a322-959cac4a8ccf admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:45.993023 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279764e-0c11-4442-a322-959cac4a8ccf admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:45.997555 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d868f-7e97-44f6-8fe0-7be6f8639b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:45.998455 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d868f-7e97-44f6-8fe0-7be6f8639b18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:45.998769 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d868f-7e97-44f6-8fe0-7be6f8639b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:45.999047 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d868f-7e97-44f6-8fe0-7be6f8639b18 admin admin] PATH_INFO: `/v3/projects/48d2b45be9be437f84941d619fd6b5d0/users/fc0c5e18c3f4435aaa9d27afcf83e29d/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:46.014656 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d868f-7e97-44f6-8fe0-7be6f8639b18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48d2b45be9be437f84941d619fd6b5d0, user_id=fc0c5e18c3f4435aaa9d27afcf83e29d, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:46.018101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d868f-7e97-44f6-8fe0-7be6f8639b18 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:46.099261 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1259/2537] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:45 2022] PUT /identity/v3/projects/48d2b45be9be437f84941d619fd6b5d0/users/fc0c5e18c3f4435aaa9d27afcf83e29d/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:46.104656 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bc426-593e-45a2-83b8-32e04099333d None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:46.111532 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bc426-593e-45a2-83b8-32e04099333d None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:46.111532 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bc426-593e-45a2-83b8-32e04099333d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:46.123605 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-912bc426-593e-45a2-83b8-32e04099333d None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:46.124794 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-912bc426-593e-45a2-83b8-32e04099333d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:46.182564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1279/2538] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:55:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:46.192615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d868f-7e97-44f6-8fe0-7be6f8639b18 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:46.194317 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-266d868f-7e97-44f6-8fe0-7be6f8639b18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:46.236865 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:46.251654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d868f-7e97-44f6-8fe0-7be6f8639b18 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:46.255289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d47ab1-0d32-4069-95a7-7356ba8a1338 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:46.257439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d47ab1-0d32-4069-95a7-7356ba8a1338 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:46.257439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d47ab1-0d32-4069-95a7-7356ba8a1338 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:46.257439 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d47ab1-0d32-4069-95a7-7356ba8a1338 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:46.265531 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d47ab1-0d32-4069-95a7-7356ba8a1338 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:46.267272 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d47ab1-0d32-4069-95a7-7356ba8a1338 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:46.276955 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1260/2539] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:47.541324 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12a287-f732-43a7-92ed-d9edb69ae7ca None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:47.541884 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12a287-f732-43a7-92ed-d9edb69ae7ca None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:47.542062 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12a287-f732-43a7-92ed-d9edb69ae7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:47.572181 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-3a12a287-f732-43a7-92ed-d9edb69ae7ca None None] MFA Rules not processed for user `fc0c5e18c3f4435aaa9d27afcf83e29d`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:47.573628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-3a12a287-f732-43a7-92ed-d9edb69ae7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:47.641028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1280/2540] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:55:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:47.645722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d47ab1-0d32-4069-95a7-7356ba8a1338 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:47.655494 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d47ab1-0d32-4069-95a7-7356ba8a1338 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:47.661117 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fedcc-4a9a-4857-9e30-160868aa1671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:47.661774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fedcc-4a9a-4857-9e30-160868aa1671 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:47.662055 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fedcc-4a9a-4857-9e30-160868aa1671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:47.662328 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fedcc-4a9a-4857-9e30-160868aa1671 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:47.663235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98fedcc-4a9a-4857-9e30-160868aa1671 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:47.664957 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98fedcc-4a9a-4857-9e30-160868aa1671 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:47.780777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1261/2541] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:47 2022] POST /identity/v3/projects => generated 371 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:47.784825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12a287-f732-43a7-92ed-d9edb69ae7ca None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:47.795498 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12a287-f732-43a7-92ed-d9edb69ae7ca None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:47.852615 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff933c7-7e20-45de-8dea-112ed6ad8d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:47.853277 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff933c7-7e20-45de-8dea-112ed6ad8d67 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:47.853560 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff933c7-7e20-45de-8dea-112ed6ad8d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:47.853833 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff933c7-7e20-45de-8dea-112ed6ad8d67 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:47.854795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff933c7-7e20-45de-8dea-112ed6ad8d67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:47.856748 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff933c7-7e20-45de-8dea-112ed6ad8d67 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:47.991326 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1281/2542] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:47 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:47.995480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fedcc-4a9a-4857-9e30-160868aa1671 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:48.006720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fedcc-4a9a-4857-9e30-160868aa1671 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:48.011249 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe85f6-6f8a-441c-b94c-eb70aae31d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:48.011906 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe85f6-6f8a-441c-b94c-eb70aae31d3b admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:48.012193 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe85f6-6f8a-441c-b94c-eb70aae31d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:48.012470 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe85f6-6f8a-441c-b94c-eb70aae31d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:48.013196 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe85f6-6f8a-441c-b94c-eb70aae31d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:48.014909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe85f6-6f8a-441c-b94c-eb70aae31d3b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:48.025849 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1262/2543] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:48.029764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff933c7-7e20-45de-8dea-112ed6ad8d67 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:48.039758 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff933c7-7e20-45de-8dea-112ed6ad8d67 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:48.044273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f00a5a-752f-44d0-9ef2-226c39fc8388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:48.045148 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f00a5a-752f-44d0-9ef2-226c39fc8388 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:48.045428 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f00a5a-752f-44d0-9ef2-226c39fc8388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:48.045703 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f00a5a-752f-44d0-9ef2-226c39fc8388 admin admin] PATH_INFO: `/v3/projects/4a6f72c0a5f347a78ec883a77c6d5d7e/users/79b28dde28364a7284affb4e1ae3aebd/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:48.057135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:48.060937 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f00a5a-752f-44d0-9ef2-226c39fc8388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a6f72c0a5f347a78ec883a77c6d5d7e, user_id=79b28dde28364a7284affb4e1ae3aebd, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:48.064237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f00a5a-752f-44d0-9ef2-226c39fc8388 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:48.225635 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1282/2544] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:48 2022] PUT /identity/v3/projects/4a6f72c0a5f347a78ec883a77c6d5d7e/users/79b28dde28364a7284affb4e1ae3aebd/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:48.229778 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe85f6-6f8a-441c-b94c-eb70aae31d3b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:48.240038 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe85f6-6f8a-441c-b94c-eb70aae31d3b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:48.296067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e8112-83b3-4107-abae-62c975ef2b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:48.296726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e8112-83b3-4107-abae-62c975ef2b6b admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:48.297006 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e8112-83b3-4107-abae-62c975ef2b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:48.297276 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e8112-83b3-4107-abae-62c975ef2b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:48.297991 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e8112-83b3-4107-abae-62c975ef2b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:48.300123 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e8112-83b3-4107-abae-62c975ef2b6b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:48.309646 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1263/2545] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:48.313739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f00a5a-752f-44d0-9ef2-226c39fc8388 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:48.324701 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f00a5a-752f-44d0-9ef2-226c39fc8388 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:48.329521 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d308eb-67ad-4c18-834b-1fb533ef0cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:48.334862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d308eb-67ad-4c18-834b-1fb533ef0cba admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:48.334862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d308eb-67ad-4c18-834b-1fb533ef0cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:48.334862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d308eb-67ad-4c18-834b-1fb533ef0cba admin admin] PATH_INFO: `/v3/projects/4a6f72c0a5f347a78ec883a77c6d5d7e/users/79b28dde28364a7284affb4e1ae3aebd/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:48.334862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d308eb-67ad-4c18-834b-1fb533ef0cba admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a6f72c0a5f347a78ec883a77c6d5d7e, user_id=79b28dde28364a7284affb4e1ae3aebd, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:48.335880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d308eb-67ad-4c18-834b-1fb533ef0cba admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:48.346864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1283/2546] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:48 2022] PUT /identity/v3/projects/4a6f72c0a5f347a78ec883a77c6d5d7e/users/79b28dde28364a7284affb4e1ae3aebd/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:49.579679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835ab90-0f78-40d8-9d8c-5678e97d2dec None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:49.580366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835ab90-0f78-40d8-9d8c-5678e97d2dec None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:49.580517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835ab90-0f78-40d8-9d8c-5678e97d2dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:49.618471 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-1835ab90-0f78-40d8-9d8c-5678e97d2dec None None] MFA Rules not processed for user `79b28dde28364a7284affb4e1ae3aebd`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:49.619727 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1835ab90-0f78-40d8-9d8c-5678e97d2dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:49.708591 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1264/2547] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:55:49 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:49.720468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d308eb-67ad-4c18-834b-1fb533ef0cba admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:49.722304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-79d308eb-67ad-4c18-834b-1fb533ef0cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:49.784486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d308eb-67ad-4c18-834b-1fb533ef0cba admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:49.788141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a63033-f344-4d67-bfd6-d85e77b93e09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:49.788779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a63033-f344-4d67-bfd6-d85e77b93e09 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:49.789169 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a63033-f344-4d67-bfd6-d85e77b93e09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:49.789444 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a63033-f344-4d67-bfd6-d85e77b93e09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:49.849519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a63033-f344-4d67-bfd6-d85e77b93e09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:49.851015 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a63033-f344-4d67-bfd6-d85e77b93e09 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:49.860041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1284/2548] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:54.730178 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835ab90-0f78-40d8-9d8c-5678e97d2dec None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:54.741048 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835ab90-0f78-40d8-9d8c-5678e97d2dec None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:54.744728 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f70085-c010-4ffc-89af-c33dde6aa9d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:54.745378 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f70085-c010-4ffc-89af-c33dde6aa9d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:54.745659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f70085-c010-4ffc-89af-c33dde6aa9d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:54.745932 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f70085-c010-4ffc-89af-c33dde6aa9d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:54.756803 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f70085-c010-4ffc-89af-c33dde6aa9d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:54.760120 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f70085-c010-4ffc-89af-c33dde6aa9d5 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:54.768113 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1265/2549] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:54 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:59.856043 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=57,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:04.763960 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:11.252843 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a63033-f344-4d67-bfd6-d85e77b93e09 service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:11.317284 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a63033-f344-4d67-bfd6-d85e77b93e09 service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:11.321709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a1358-e07c-4b0f-b764-da50ebc01487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:11.322607 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a1358-e07c-4b0f-b764-da50ebc01487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:11.322896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a1358-e07c-4b0f-b764-da50ebc01487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:11.323173 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a1358-e07c-4b0f-b764-da50ebc01487 admin admin] PATH_INFO: `/v3/users/fc0c5e18c3f4435aaa9d27afcf83e29d` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:11.324470 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a1358-e07c-4b0f-b764-da50ebc01487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc0c5e18c3f4435aaa9d27afcf83e29d)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:11.326389 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a1358-e07c-4b0f-b764-da50ebc01487 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:11.519069 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-b19a1358-e07c-4b0f-b764-da50ebc01487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc0c5e18c3f4435aaa9d27afcf83e29d'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:11.525698 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-b19a1358-e07c-4b0f-b764-da50ebc01487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc0c5e18c3f4435aaa9d27afcf83e29d'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:11.526346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-b19a1358-e07c-4b0f-b764-da50ebc01487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc0c5e18c3f4435aaa9d27afcf83e29d'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:11.528066 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-b19a1358-e07c-4b0f-b764-da50ebc01487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc0c5e18c3f4435aaa9d27afcf83e29d'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:11.539440 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-b19a1358-e07c-4b0f-b764-da50ebc01487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc0c5e18c3f4435aaa9d27afcf83e29d'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:11.630277 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1285/2550] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:56:11 2022] DELETE /identity/v3/users/fc0c5e18c3f4435aaa9d27afcf83e29d => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:11.854364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f70085-c010-4ffc-89af-c33dde6aa9d5 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:11.865900 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f70085-c010-4ffc-89af-c33dde6aa9d5 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:11.924260 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04cf22-bab4-4eac-96a7-fc180c66a61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:11.925186 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04cf22-bab4-4eac-96a7-fc180c66a61d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:11.925466 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04cf22-bab4-4eac-96a7-fc180c66a61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:11.925738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04cf22-bab4-4eac-96a7-fc180c66a61d admin admin] PATH_INFO: `/v3/projects/48d2b45be9be437f84941d619fd6b5d0` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:11.926934 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04cf22-bab4-4eac-96a7-fc180c66a61d admin admin] RBAC: Authorizing `identity:delete_project(project_id=48d2b45be9be437f84941d619fd6b5d0)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:11.928890 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04cf22-bab4-4eac-96a7-fc180c66a61d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:12.025797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cf04cf22-bab4-4eac-96a7-fc180c66a61d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48d2b45be9be437f84941d619fd6b5d0', 'request_id': 'req-cf04cf22-bab4-4eac-96a7-fc180c66a61d'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:12.032341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cf04cf22-bab4-4eac-96a7-fc180c66a61d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48d2b45be9be437f84941d619fd6b5d0', 'request_id': 'req-cf04cf22-bab4-4eac-96a7-fc180c66a61d'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:12.098892 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cf04cf22-bab4-4eac-96a7-fc180c66a61d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48d2b45be9be437f84941d619fd6b5d0', 'request_id': 'req-cf04cf22-bab4-4eac-96a7-fc180c66a61d'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:12.111541 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cf04cf22-bab4-4eac-96a7-fc180c66a61d admin admin] The token cache is being invalidate because project 48d2b45be9be437f84941d619fd6b5d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:56:12.111541 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cf04cf22-bab4-4eac-96a7-fc180c66a61d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:12.111756 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cf04cf22-bab4-4eac-96a7-fc180c66a61d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:12.113154 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1266/2551] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:56:11 2022] DELETE /identity/v3/projects/48d2b45be9be437f84941d619fd6b5d0 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:12.118028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a1358-e07c-4b0f-b764-da50ebc01487 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:12.121592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b19a1358-e07c-4b0f-b764-da50ebc01487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:12.133756 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a1358-e07c-4b0f-b764-da50ebc01487 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:12.191730 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a3b21-4fa6-4004-830d-654b57d4ceed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:12.192596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a3b21-4fa6-4004-830d-654b57d4ceed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:12.192873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a3b21-4fa6-4004-830d-654b57d4ceed admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:12.193144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a3b21-4fa6-4004-830d-654b57d4ceed admin admin] PATH_INFO: `/v3/users/79b28dde28364a7284affb4e1ae3aebd` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:12.194308 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a3b21-4fa6-4004-830d-654b57d4ceed admin admin] RBAC: Authorizing `identity:delete_user(user_id=79b28dde28364a7284affb4e1ae3aebd)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:12.196253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a3b21-4fa6-4004-830d-654b57d4ceed admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:12.218937 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:12.386678 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-8a2a3b21-4fa6-4004-830d-654b57d4ceed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79b28dde28364a7284affb4e1ae3aebd'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:12.394033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-8a2a3b21-4fa6-4004-830d-654b57d4ceed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79b28dde28364a7284affb4e1ae3aebd'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:12.402892 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-8a2a3b21-4fa6-4004-830d-654b57d4ceed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79b28dde28364a7284affb4e1ae3aebd'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:12.402892 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-8a2a3b21-4fa6-4004-830d-654b57d4ceed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79b28dde28364a7284affb4e1ae3aebd'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:12.406985 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-8a2a3b21-4fa6-4004-830d-654b57d4ceed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79b28dde28364a7284affb4e1ae3aebd'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:12.470923 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1286/2552] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:56:12 2022] DELETE /identity/v3/users/79b28dde28364a7284affb4e1ae3aebd => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:56:12.711526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04cf22-bab4-4eac-96a7-fc180c66a61d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:12.712955 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-cf04cf22-bab4-4eac-96a7-fc180c66a61d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:12.727809 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04cf22-bab4-4eac-96a7-fc180c66a61d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:12.786187 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a83491-5e49-43e1-b446-3ac664c44825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:12.787154 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a83491-5e49-43e1-b446-3ac664c44825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:12.787442 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a83491-5e49-43e1-b446-3ac664c44825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:12.787718 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a83491-5e49-43e1-b446-3ac664c44825 admin admin] PATH_INFO: `/v3/projects/4a6f72c0a5f347a78ec883a77c6d5d7e` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:12.788898 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a83491-5e49-43e1-b446-3ac664c44825 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a6f72c0a5f347a78ec883a77c6d5d7e)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:12.790888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a83491-5e49-43e1-b446-3ac664c44825 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:12.910926 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:12.922770 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-53a83491-5e49-43e1-b446-3ac664c44825 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a6f72c0a5f347a78ec883a77c6d5d7e', 'request_id': 'req-53a83491-5e49-43e1-b446-3ac664c44825'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:12.928986 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-53a83491-5e49-43e1-b446-3ac664c44825 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a6f72c0a5f347a78ec883a77c6d5d7e', 'request_id': 'req-53a83491-5e49-43e1-b446-3ac664c44825'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:13.000991 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-53a83491-5e49-43e1-b446-3ac664c44825 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a6f72c0a5f347a78ec883a77c6d5d7e', 'request_id': 'req-53a83491-5e49-43e1-b446-3ac664c44825'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:13.012819 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-53a83491-5e49-43e1-b446-3ac664c44825 admin admin] The token cache is being invalidate because project 4a6f72c0a5f347a78ec883a77c6d5d7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:56:13.013097 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-53a83491-5e49-43e1-b446-3ac664c44825 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:13.013717 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-53a83491-5e49-43e1-b446-3ac664c44825 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:13.015324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1267/2553] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:56:12 2022] DELETE /identity/v3/projects/4a6f72c0a5f347a78ec883a77c6d5d7e => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:13.020293 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a055e7-9916-42f6-933b-095ab4ff7869 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:13.020680 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a055e7-9916-42f6-933b-095ab4ff7869 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:13.021031 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a055e7-9916-42f6-933b-095ab4ff7869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:13.039584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-21a055e7-9916-42f6-933b-095ab4ff7869 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:13.041215 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-21a055e7-9916-42f6-933b-095ab4ff7869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:13.098661 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1287/2554] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:56:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:13.107000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1c875-e148-43fe-b533-8f2b0464da6c None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:13.107291 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1c875-e148-43fe-b533-8f2b0464da6c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:13.107569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1c875-e148-43fe-b533-8f2b0464da6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:13.125549 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-97c1c875-e148-43fe-b533-8f2b0464da6c None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:13.126814 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-97c1c875-e148-43fe-b533-8f2b0464da6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:13.134573 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1268/2555] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:56:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:13.139260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a055e7-9916-42f6-933b-095ab4ff7869 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:13.148640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a055e7-9916-42f6-933b-095ab4ff7869 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:13.154205 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e5623-dc4c-4233-8e09-dad4c7aea529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:13.154905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e5623-dc4c-4233-8e09-dad4c7aea529 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:13.155183 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e5623-dc4c-4233-8e09-dad4c7aea529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:13.155452 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e5623-dc4c-4233-8e09-dad4c7aea529 admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:13.156198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e5623-dc4c-4233-8e09-dad4c7aea529 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:56:13.156503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e5623-dc4c-4233-8e09-dad4c7aea529 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:13.158172 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e5623-dc4c-4233-8e09-dad4c7aea529 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:13.169623 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1288/2556] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:56:13.173858 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1c875-e148-43fe-b533-8f2b0464da6c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:13.183089 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1c875-e148-43fe-b533-8f2b0464da6c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:13.188142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c7ad3-a317-42da-93f3-f2c1b2456142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:13.188778 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c7ad3-a317-42da-93f3-f2c1b2456142 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:13.189056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c7ad3-a317-42da-93f3-f2c1b2456142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:13.189326 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c7ad3-a317-42da-93f3-f2c1b2456142 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:13.190017 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206c7ad3-a317-42da-93f3-f2c1b2456142 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:13.192279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206c7ad3-a317-42da-93f3-f2c1b2456142 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:13.300306 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1269/2557] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:56:13 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:56:13.304322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e5623-dc4c-4233-8e09-dad4c7aea529 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:13.314631 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e5623-dc4c-4233-8e09-dad4c7aea529 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:13.369895 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cede41a-76c4-4c72-8c72-63bba7ff0f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:13.371182 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cede41a-76c4-4c72-8c72-63bba7ff0f3a admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:13.371669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cede41a-76c4-4c72-8c72-63bba7ff0f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:13.371782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cede41a-76c4-4c72-8c72-63bba7ff0f3a admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:13.372386 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cede41a-76c4-4c72-8c72-63bba7ff0f3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:13.374299 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cede41a-76c4-4c72-8c72-63bba7ff0f3a admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:13.441843 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1289/2558] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:56:13 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:56:13.446012 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c7ad3-a317-42da-93f3-f2c1b2456142 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:13.456915 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c7ad3-a317-42da-93f3-f2c1b2456142 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:13.462199 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b2ea6-718b-417e-8fa0-458b2195960d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:13.462902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b2ea6-718b-417e-8fa0-458b2195960d admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:13.463187 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b2ea6-718b-417e-8fa0-458b2195960d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:13.463466 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b2ea6-718b-417e-8fa0-458b2195960d admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:13.464228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b2ea6-718b-417e-8fa0-458b2195960d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:13.465914 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b2ea6-718b-417e-8fa0-458b2195960d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:13.476413 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1270/2559] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:56:13.486138 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cede41a-76c4-4c72-8c72-63bba7ff0f3a admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:13.496891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cede41a-76c4-4c72-8c72-63bba7ff0f3a admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:13.501553 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e37645-e90a-4c6d-86f3-4ffeed0530e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:13.502205 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e37645-e90a-4c6d-86f3-4ffeed0530e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:13.502490 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e37645-e90a-4c6d-86f3-4ffeed0530e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:13.502800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e37645-e90a-4c6d-86f3-4ffeed0530e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:13.503542 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e37645-e90a-4c6d-86f3-4ffeed0530e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:13.505132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e37645-e90a-4c6d-86f3-4ffeed0530e0 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:13.515779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1290/2560] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:56:13.519852 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b2ea6-718b-417e-8fa0-458b2195960d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:13.530301 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b2ea6-718b-417e-8fa0-458b2195960d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:13.534907 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267c4cd-b895-4c52-a38f-adf34c56796a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:13.535774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267c4cd-b895-4c52-a38f-adf34c56796a admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:13.536056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267c4cd-b895-4c52-a38f-adf34c56796a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:13.536327 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267c4cd-b895-4c52-a38f-adf34c56796a admin admin] PATH_INFO: `/v3/projects/a1788128bcf74326b11a18657ea5e470/users/de7ed2b717544f1483831696ff15b821/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:13.551880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267c4cd-b895-4c52-a38f-adf34c56796a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1788128bcf74326b11a18657ea5e470, user_id=de7ed2b717544f1483831696ff15b821, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:13.554518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267c4cd-b895-4c52-a38f-adf34c56796a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:13.613284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1271/2561] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:56:13 2022] PUT /identity/v3/projects/a1788128bcf74326b11a18657ea5e470/users/de7ed2b717544f1483831696ff15b821/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:13.625081 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0012f-3b17-4e5a-bd3a-3d5c7c785a9c None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:13.625371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0012f-3b17-4e5a-bd3a-3d5c7c785a9c None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:13.625676 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0012f-3b17-4e5a-bd3a-3d5c7c785a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:13.632420 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:13.654288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-b5b0012f-3b17-4e5a-bd3a-3d5c7c785a9c None None] MFA Rules not processed for user `de7ed2b717544f1483831696ff15b821`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:13.655597 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b5b0012f-3b17-4e5a-bd3a-3d5c7c785a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:13.722864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1291/2562] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:56:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:13.734580 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267c4cd-b895-4c52-a38f-adf34c56796a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:13.736263 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4267c4cd-b895-4c52-a38f-adf34c56796a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:13.793033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267c4cd-b895-4c52-a38f-adf34c56796a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:13.796678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b8a74-09b6-4ed2-aaaf-506f5dda5d48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:13.797323 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b8a74-09b6-4ed2-aaaf-506f5dda5d48 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:13.797609 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b8a74-09b6-4ed2-aaaf-506f5dda5d48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:13.797887 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b8a74-09b6-4ed2-aaaf-506f5dda5d48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:13.806118 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b8a74-09b6-4ed2-aaaf-506f5dda5d48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:13.807589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b8a74-09b6-4ed2-aaaf-506f5dda5d48 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:13.816986 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1272/2563] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:56:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:56:23.719851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=30 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:23.812980 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:28.413697 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0012f-3b17-4e5a-bd3a-3d5c7c785a9c None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:28.415240 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b5b0012f-3b17-4e5a-bd3a-3d5c7c785a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:28.510162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0012f-3b17-4e5a-bd3a-3d5c7c785a9c None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:28.518029 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d9403-7b50-4457-aef4-797562cd549b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:28.518734 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d9403-7b50-4457-aef4-797562cd549b service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:28.519027 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d9403-7b50-4457-aef4-797562cd549b service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:28.519307 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d9403-7b50-4457-aef4-797562cd549b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:28.521490 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-595d9403-7b50-4457-aef4-797562cd549b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:28.542049 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d9403-7b50-4457-aef4-797562cd549b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:28.543713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d9403-7b50-4457-aef4-797562cd549b service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:28.557930 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1292/2564] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:56:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:56:38.553288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=34 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:49.491235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b8a74-09b6-4ed2-aaaf-506f5dda5d48 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:49.512834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b8a74-09b6-4ed2-aaaf-506f5dda5d48 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:49.517771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7df6d-ec98-47ab-9edb-d15d57238ac9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:49.518825 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7df6d-ec98-47ab-9edb-d15d57238ac9 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:49.519305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7df6d-ec98-47ab-9edb-d15d57238ac9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:49.519750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7df6d-ec98-47ab-9edb-d15d57238ac9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:49.521749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bbf7df6d-ec98-47ab-9edb-d15d57238ac9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:49.629308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7df6d-ec98-47ab-9edb-d15d57238ac9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:49.631431 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7df6d-ec98-47ab-9edb-d15d57238ac9 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:49.654817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1273/2565] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:56:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:56:49.808421 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d9403-7b50-4457-aef4-797562cd549b service nova] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:49.809877 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-595d9403-7b50-4457-aef4-797562cd549b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:49.953330 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d9403-7b50-4457-aef4-797562cd549b service nova] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:49.957674 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f50dfd-1195-4749-990c-a4f5cdc07dda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:49.958319 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f50dfd-1195-4749-990c-a4f5cdc07dda service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:49.958649 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f50dfd-1195-4749-990c-a4f5cdc07dda service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:49.958931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f50dfd-1195-4749-990c-a4f5cdc07dda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:49.960720 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-65f50dfd-1195-4749-990c-a4f5cdc07dda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:50.053594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f50dfd-1195-4749-990c-a4f5cdc07dda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:50.055116 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f50dfd-1195-4749-990c-a4f5cdc07dda service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:50.080090 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1293/2566] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:56:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:56:52.466103 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7df6d-ec98-47ab-9edb-d15d57238ac9 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:52.477981 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7df6d-ec98-47ab-9edb-d15d57238ac9 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:52.490947 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c723ab7-2545-4e32-9ff4-f050b40b2ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:52.491853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c723ab7-2545-4e32-9ff4-f050b40b2ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:52.492139 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c723ab7-2545-4e32-9ff4-f050b40b2ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:52.494607 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c723ab7-2545-4e32-9ff4-f050b40b2ce4 admin admin] PATH_INFO: `/v3/users/de7ed2b717544f1483831696ff15b821` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:52.496055 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c723ab7-2545-4e32-9ff4-f050b40b2ce4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de7ed2b717544f1483831696ff15b821)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:52.498229 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c723ab7-2545-4e32-9ff4-f050b40b2ce4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:52.748393 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9c723ab7-2545-4e32-9ff4-f050b40b2ce4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de7ed2b717544f1483831696ff15b821'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:52.758702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9c723ab7-2545-4e32-9ff4-f050b40b2ce4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de7ed2b717544f1483831696ff15b821'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:52.759704 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9c723ab7-2545-4e32-9ff4-f050b40b2ce4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de7ed2b717544f1483831696ff15b821'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:52.761040 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9c723ab7-2545-4e32-9ff4-f050b40b2ce4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de7ed2b717544f1483831696ff15b821'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:52.782231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9c723ab7-2545-4e32-9ff4-f050b40b2ce4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de7ed2b717544f1483831696ff15b821'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:52.979142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1274/2567] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:56:52 2022] DELETE /identity/v3/users/de7ed2b717544f1483831696ff15b821 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:52.984120 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b23f3-ba7a-4407-98ba-78b09f930b77 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:52.984613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b23f3-ba7a-4407-98ba-78b09f930b77 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:52.984840 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b23f3-ba7a-4407-98ba-78b09f930b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:53.042213 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-e45b23f3-ba7a-4407-98ba-78b09f930b77 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:53.047404 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e45b23f3-ba7a-4407-98ba-78b09f930b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:53.078745 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:53.147218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1294/2568] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:56:53.185839 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c723ab7-2545-4e32-9ff4-f050b40b2ce4 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:53.193789 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9c723ab7-2545-4e32-9ff4-f050b40b2ce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:53.238142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:53.386865 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c723ab7-2545-4e32-9ff4-f050b40b2ce4 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:53.386865 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a6a8e-5545-44c7-a4d3-f98a94d0dd28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:53.386865 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a6a8e-5545-44c7-a4d3-f98a94d0dd28 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:53.386865 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a6a8e-5545-44c7-a4d3-f98a94d0dd28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:53.387858 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a6a8e-5545-44c7-a4d3-f98a94d0dd28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:53.401486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a6a8e-5545-44c7-a4d3-f98a94d0dd28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:53.404741 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a6a8e-5545-44c7-a4d3-f98a94d0dd28 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:53.433355 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1275/2569] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:56:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:56:53.776304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b23f3-ba7a-4407-98ba-78b09f930b77 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:53.778595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e45b23f3-ba7a-4407-98ba-78b09f930b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:53.799006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b23f3-ba7a-4407-98ba-78b09f930b77 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:53.811184 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b63d7-167e-4ec8-be7c-a22e48857610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:53.812385 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b63d7-167e-4ec8-be7c-a22e48857610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:53.812676 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b63d7-167e-4ec8-be7c-a22e48857610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:53.813105 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b63d7-167e-4ec8-be7c-a22e48857610 admin admin] PATH_INFO: `/v3/users/c2da36c53e064fedb65ab202d265eb12` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:53.814571 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b63d7-167e-4ec8-be7c-a22e48857610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2da36c53e064fedb65ab202d265eb12)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:53.818863 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b63d7-167e-4ec8-be7c-a22e48857610 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:54.050615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a6a8e-5545-44c7-a4d3-f98a94d0dd28 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:54.052782 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-982a6a8e-5545-44c7-a4d3-f98a94d0dd28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:54.068431 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a6a8e-5545-44c7-a4d3-f98a94d0dd28 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:54.074964 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f5858-fb20-491f-acf4-9dbc26f8e7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:54.076813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f5858-fb20-491f-acf4-9dbc26f8e7b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:54.077765 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f5858-fb20-491f-acf4-9dbc26f8e7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:54.078131 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f5858-fb20-491f-acf4-9dbc26f8e7b3 admin admin] PATH_INFO: `/v3/projects/a1788128bcf74326b11a18657ea5e470` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:54.079606 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f5858-fb20-491f-acf4-9dbc26f8e7b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1788128bcf74326b11a18657ea5e470)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:54.081959 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f5858-fb20-491f-acf4-9dbc26f8e7b3 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:54.179191 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fe0b63d7-167e-4ec8-be7c-a22e48857610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2da36c53e064fedb65ab202d265eb12'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.188896 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fe0b63d7-167e-4ec8-be7c-a22e48857610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2da36c53e064fedb65ab202d265eb12'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.194781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fe0b63d7-167e-4ec8-be7c-a22e48857610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2da36c53e064fedb65ab202d265eb12'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.194781 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fe0b63d7-167e-4ec8-be7c-a22e48857610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2da36c53e064fedb65ab202d265eb12'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.204613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fe0b63d7-167e-4ec8-be7c-a22e48857610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2da36c53e064fedb65ab202d265eb12'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.235851 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7c9f5858-fb20-491f-acf4-9dbc26f8e7b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1788128bcf74326b11a18657ea5e470', 'request_id': 'req-7c9f5858-fb20-491f-acf4-9dbc26f8e7b3'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.258074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7c9f5858-fb20-491f-acf4-9dbc26f8e7b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1788128bcf74326b11a18657ea5e470', 'request_id': 'req-7c9f5858-fb20-491f-acf4-9dbc26f8e7b3'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.294778 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1295/2570] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:56:53 2022] DELETE /identity/v3/users/c2da36c53e064fedb65ab202d265eb12 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:54.300061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e221d-89d1-4d78-9bc8-b9a9dae8dc91 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:54.300426 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e221d-89d1-4d78-9bc8-b9a9dae8dc91 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:54.300954 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e221d-89d1-4d78-9bc8-b9a9dae8dc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:54.325649 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-ff6e221d-89d1-4d78-9bc8-b9a9dae8dc91 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:54.326967 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-ff6e221d-89d1-4d78-9bc8-b9a9dae8dc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:54.379943 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7c9f5858-fb20-491f-acf4-9dbc26f8e7b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1788128bcf74326b11a18657ea5e470', 'request_id': 'req-7c9f5858-fb20-491f-acf4-9dbc26f8e7b3'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.411243 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7c9f5858-fb20-491f-acf4-9dbc26f8e7b3 admin admin] The token cache is being invalidate because project a1788128bcf74326b11a18657ea5e470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:56:54.411651 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7c9f5858-fb20-491f-acf4-9dbc26f8e7b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.419156 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7c9f5858-fb20-491f-acf4-9dbc26f8e7b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.420367 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1276/2571] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:56:54 2022] DELETE /identity/v3/projects/a1788128bcf74326b11a18657ea5e470 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:56:54.426328 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9206d3-b0ed-4576-8b01-a120e6c5323c None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:54.426746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9206d3-b0ed-4576-8b01-a120e6c5323c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:54.427116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9206d3-b0ed-4576-8b01-a120e6c5323c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:54.458447 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1296/2572] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:54.471215 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e221d-89d1-4d78-9bc8-b9a9dae8dc91 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:54.473765 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-ff6e221d-89d1-4d78-9bc8-b9a9dae8dc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:54.524729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:54.526485 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-6d9206d3-b0ed-4576-8b01-a120e6c5323c None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:54.527746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6d9206d3-b0ed-4576-8b01-a120e6c5323c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:54.539070 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1277/2573] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:56:54.548557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46a289-ae64-4a95-bfa1-38fcc58d4dd0 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:54.549183 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46a289-ae64-4a95-bfa1-38fcc58d4dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:54.549588 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46a289-ae64-4a95-bfa1-38fcc58d4dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:54.566717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e221d-89d1-4d78-9bc8-b9a9dae8dc91 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:54.573862 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-2a46a289-ae64-4a95-bfa1-38fcc58d4dd0 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:54.575095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2a46a289-ae64-4a95-bfa1-38fcc58d4dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:54.576186 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7105e-1938-4032-87b1-4ac5835ee7ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:54.577697 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7105e-1938-4032-87b1-4ac5835ee7ab service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:54.578369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7105e-1938-4032-87b1-4ac5835ee7ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:54.578904 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7105e-1938-4032-87b1-4ac5835ee7ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:54.590689 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1278/2574] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:54.593481 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7105e-1938-4032-87b1-4ac5835ee7ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:54.599411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7105e-1938-4032-87b1-4ac5835ee7ab service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:54.602065 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46a289-ae64-4a95-bfa1-38fcc58d4dd0 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:54.608681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1297/2575] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:56:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:56:54.616278 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46a289-ae64-4a95-bfa1-38fcc58d4dd0 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:54.621686 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c6887-fcda-402a-b212-8ea4a3c0751a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:54.622363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c6887-fcda-402a-b212-8ea4a3c0751a admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:54.622875 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c6887-fcda-402a-b212-8ea4a3c0751a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:54.623306 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c6887-fcda-402a-b212-8ea4a3c0751a admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:54.624966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c6887-fcda-402a-b212-8ea4a3c0751a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:56:54.625435 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c6887-fcda-402a-b212-8ea4a3c0751a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:54.627842 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c6887-fcda-402a-b212-8ea4a3c0751a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:54.643093 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1279/2576] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:56:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:56:54.648134 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7105e-1938-4032-87b1-4ac5835ee7ab service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:54.659090 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7105e-1938-4032-87b1-4ac5835ee7ab service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:54.671905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f3981-d76f-4079-aa83-d3299fc33d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:54.671905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f3981-d76f-4079-aa83-d3299fc33d97 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:54.671905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f3981-d76f-4079-aa83-d3299fc33d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:54.671905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f3981-d76f-4079-aa83-d3299fc33d97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:54.672574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f3981-d76f-4079-aa83-d3299fc33d97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:54.672574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f3981-d76f-4079-aa83-d3299fc33d97 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:54.821182 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1298/2577] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:56:54 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:56:54.825442 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c6887-fcda-402a-b212-8ea4a3c0751a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:54.835737 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c6887-fcda-402a-b212-8ea4a3c0751a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:54.883194 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:54.903265 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a1345-8283-4acb-af1b-73be798a36aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:54.904240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a1345-8283-4acb-af1b-73be798a36aa admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:54.904460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a1345-8283-4acb-af1b-73be798a36aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:54.904937 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a1345-8283-4acb-af1b-73be798a36aa admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:54.905964 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a1345-8283-4acb-af1b-73be798a36aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:54.908234 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a1345-8283-4acb-af1b-73be798a36aa admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:55.039699 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1280/2578] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:56:54 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:56:55.041140 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f3981-d76f-4079-aa83-d3299fc33d97 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:55.065761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f3981-d76f-4079-aa83-d3299fc33d97 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:55.065761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276347e-49d5-43d8-bb67-8773e4d1131b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:55.065761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276347e-49d5-43d8-bb67-8773e4d1131b admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.065761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276347e-49d5-43d8-bb67-8773e4d1131b admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.066140 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276347e-49d5-43d8-bb67-8773e4d1131b admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.066140 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276347e-49d5-43d8-bb67-8773e4d1131b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:55.066444 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276347e-49d5-43d8-bb67-8773e4d1131b admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:55.099394 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1299/2579] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:56:55.101948 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a1345-8283-4acb-af1b-73be798a36aa admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:55.114357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a1345-8283-4acb-af1b-73be798a36aa admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:55.119640 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97a39b-a798-4de8-be93-b64245febdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:55.120303 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97a39b-a798-4de8-be93-b64245febdd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.120753 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97a39b-a798-4de8-be93-b64245febdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.121030 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97a39b-a798-4de8-be93-b64245febdd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.121754 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97a39b-a798-4de8-be93-b64245febdd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:55.123401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97a39b-a798-4de8-be93-b64245febdd0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:55.135609 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1281/2580] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:56:55.139979 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276347e-49d5-43d8-bb67-8773e4d1131b admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:55.150405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276347e-49d5-43d8-bb67-8773e4d1131b admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:55.156024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce57890-d5dc-4159-8f70-b26d0fb5352a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:55.157396 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce57890-d5dc-4159-8f70-b26d0fb5352a admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.157767 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce57890-d5dc-4159-8f70-b26d0fb5352a admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.158110 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce57890-d5dc-4159-8f70-b26d0fb5352a admin admin] PATH_INFO: `/v3/projects/a8df00f506154d1ca1d9da3e28b56712/users/9040cb92bc354fbd985b40db56894042/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.177083 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97a39b-a798-4de8-be93-b64245febdd0 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:55.178249 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce57890-d5dc-4159-8f70-b26d0fb5352a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8df00f506154d1ca1d9da3e28b56712, user_id=9040cb92bc354fbd985b40db56894042, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:55.179488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-0d97a39b-a798-4de8-be93-b64245febdd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:55.181626 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce57890-d5dc-4159-8f70-b26d0fb5352a admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:55.209571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97a39b-a798-4de8-be93-b64245febdd0 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:55.214482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bfc8be-8c24-4da6-83b9-4ffe4de6c034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:55.215406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfc8be-8c24-4da6-83b9-4ffe4de6c034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.215771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfc8be-8c24-4da6-83b9-4ffe4de6c034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.216037 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfc8be-8c24-4da6-83b9-4ffe4de6c034 admin admin] PATH_INFO: `/v3/projects/6dab0670bac3443abac22ad78e103076` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.218134 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bfc8be-8c24-4da6-83b9-4ffe4de6c034 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dab0670bac3443abac22ad78e103076)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:55.220125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bfc8be-8c24-4da6-83b9-4ffe4de6c034 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:55.266354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1300/2581] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:56:55 2022] PUT /identity/v3/projects/a8df00f506154d1ca1d9da3e28b56712/users/9040cb92bc354fbd985b40db56894042/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:55.277526 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ce4a1-3053-4d51-aa76-7db3a7397075 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.278044 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ce4a1-3053-4d51-aa76-7db3a7397075 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.278319 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ce4a1-3053-4d51-aa76-7db3a7397075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.306327 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-819ce4a1-3053-4d51-aa76-7db3a7397075 None None] MFA Rules not processed for user `9040cb92bc354fbd985b40db56894042`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:55.312212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-819ce4a1-3053-4d51-aa76-7db3a7397075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:55.364326 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c0bfc8be-8c24-4da6-83b9-4ffe4de6c034 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dab0670bac3443abac22ad78e103076', 'request_id': 'req-c0bfc8be-8c24-4da6-83b9-4ffe4de6c034'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:55.370723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c0bfc8be-8c24-4da6-83b9-4ffe4de6c034 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dab0670bac3443abac22ad78e103076', 'request_id': 'req-c0bfc8be-8c24-4da6-83b9-4ffe4de6c034'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:55.378436 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1301/2582] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:56:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:56:55.389608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ce4a1-3053-4d51-aa76-7db3a7397075 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:55.391281 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-819ce4a1-3053-4d51-aa76-7db3a7397075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:55.450582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ce4a1-3053-4d51-aa76-7db3a7397075 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:55.466121 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c0bfc8be-8c24-4da6-83b9-4ffe4de6c034 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dab0670bac3443abac22ad78e103076', 'request_id': 'req-c0bfc8be-8c24-4da6-83b9-4ffe4de6c034'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:55.480630 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c0bfc8be-8c24-4da6-83b9-4ffe4de6c034 admin admin] The token cache is being invalidate because project 6dab0670bac3443abac22ad78e103076 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:56:55.481131 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c0bfc8be-8c24-4da6-83b9-4ffe4de6c034 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:55.489470 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c0bfc8be-8c24-4da6-83b9-4ffe4de6c034 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:55.491588 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1282/2583] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:56:55 2022] DELETE /identity/v3/projects/6dab0670bac3443abac22ad78e103076 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:56:55.495637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bfc8be-8c24-4da6-83b9-4ffe4de6c034 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:55.498048 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c0bfc8be-8c24-4da6-83b9-4ffe4de6c034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:55.522116 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fcfb9-a1cb-4934-bc8a-2dc5885fb42d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:55.522914 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fcfb9-a1cb-4934-bc8a-2dc5885fb42d service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.523270 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fcfb9-a1cb-4934-bc8a-2dc5885fb42d service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.523618 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fcfb9-a1cb-4934-bc8a-2dc5885fb42d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.526851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f76fcfb9-a1cb-4934-bc8a-2dc5885fb42d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:55.567613 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bfc8be-8c24-4da6-83b9-4ffe4de6c034 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:55.573310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067bc34-ca2d-4228-b09e-439fe14e588b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:55.574362 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067bc34-ca2d-4228-b09e-439fe14e588b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.574681 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067bc34-ca2d-4228-b09e-439fe14e588b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.574963 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067bc34-ca2d-4228-b09e-439fe14e588b admin admin] PATH_INFO: `/v3/users/640f83e986cd4eb998349f50535ef7ce` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.576183 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067bc34-ca2d-4228-b09e-439fe14e588b admin admin] RBAC: Authorizing `identity:delete_user(user_id=640f83e986cd4eb998349f50535ef7ce)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:55.579806 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067bc34-ca2d-4228-b09e-439fe14e588b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:55.596067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:55.612194 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76fcfb9-a1cb-4934-bc8a-2dc5885fb42d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:55.614323 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76fcfb9-a1cb-4934-bc8a-2dc5885fb42d service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:55.626184 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1302/2584] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:56:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:56:55.829494 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4067bc34-ca2d-4228-b09e-439fe14e588b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '640f83e986cd4eb998349f50535ef7ce'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:55.840511 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4067bc34-ca2d-4228-b09e-439fe14e588b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '640f83e986cd4eb998349f50535ef7ce'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:55.842055 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4067bc34-ca2d-4228-b09e-439fe14e588b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '640f83e986cd4eb998349f50535ef7ce'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:55.842055 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4067bc34-ca2d-4228-b09e-439fe14e588b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '640f83e986cd4eb998349f50535ef7ce'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:55.855668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4067bc34-ca2d-4228-b09e-439fe14e588b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '640f83e986cd4eb998349f50535ef7ce'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:55.981405 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1283/2585] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:56:55 2022] DELETE /identity/v3/users/640f83e986cd4eb998349f50535ef7ce => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:56:56.273530 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=82,keystone:INSERT=4,keystone:DELETE=14 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:56.846731 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fcfb9-a1cb-4934-bc8a-2dc5885fb42d service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:56.857738 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fcfb9-a1cb-4934-bc8a-2dc5885fb42d service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:56.948906 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e80b51-00ac-4b65-8564-5ef515870f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:56.950019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e80b51-00ac-4b65-8564-5ef515870f4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:56.950430 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e80b51-00ac-4b65-8564-5ef515870f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:56.950848 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e80b51-00ac-4b65-8564-5ef515870f4a admin admin] PATH_INFO: `/v3/projects/8e12a14669a84ee4a76bddfb3fb84ec2` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:56.952397 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e80b51-00ac-4b65-8564-5ef515870f4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e12a14669a84ee4a76bddfb3fb84ec2)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:56.958655 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e80b51-00ac-4b65-8564-5ef515870f4a admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:57.096777 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-04e80b51-00ac-4b65-8564-5ef515870f4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e12a14669a84ee4a76bddfb3fb84ec2', 'request_id': 'req-04e80b51-00ac-4b65-8564-5ef515870f4a'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:57.104183 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-04e80b51-00ac-4b65-8564-5ef515870f4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e12a14669a84ee4a76bddfb3fb84ec2', 'request_id': 'req-04e80b51-00ac-4b65-8564-5ef515870f4a'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:57.173930 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-04e80b51-00ac-4b65-8564-5ef515870f4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e12a14669a84ee4a76bddfb3fb84ec2', 'request_id': 'req-04e80b51-00ac-4b65-8564-5ef515870f4a'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:57.187369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-04e80b51-00ac-4b65-8564-5ef515870f4a admin admin] The token cache is being invalidate because project 8e12a14669a84ee4a76bddfb3fb84ec2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:56:57.189765 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-04e80b51-00ac-4b65-8564-5ef515870f4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:57.189765 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-04e80b51-00ac-4b65-8564-5ef515870f4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:57.189977 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1303/2586] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:56:56 2022] DELETE /identity/v3/projects/8e12a14669a84ee4a76bddfb3fb84ec2 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:57.195181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbf21b-3907-4664-b435-21197b00b8c4 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:57.195465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbf21b-3907-4664-b435-21197b00b8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:57.195750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbf21b-3907-4664-b435-21197b00b8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:57.215887 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-35dbf21b-3907-4664-b435-21197b00b8c4 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:57.216334 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-35dbf21b-3907-4664-b435-21197b00b8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:57.283853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1284/2587] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:57.293017 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ec9ab-4427-423d-810a-26eb5870377f None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:57.293411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ec9ab-4427-423d-810a-26eb5870377f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:57.293628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ec9ab-4427-423d-810a-26eb5870377f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:57.314211 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-489ec9ab-4427-423d-810a-26eb5870377f None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:57.315717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-489ec9ab-4427-423d-810a-26eb5870377f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:57.324306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1304/2588] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:57.329146 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbf21b-3907-4664-b435-21197b00b8c4 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:57.339059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbf21b-3907-4664-b435-21197b00b8c4 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:57.344716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3341b2-4ac8-47f1-b688-a412f5b8a4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:57.345409 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3341b2-4ac8-47f1-b688-a412f5b8a4ae admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:57.345695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3341b2-4ac8-47f1-b688-a412f5b8a4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:57.345978 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3341b2-4ac8-47f1-b688-a412f5b8a4ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:57.348080 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3341b2-4ac8-47f1-b688-a412f5b8a4ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:56:57.348391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3341b2-4ac8-47f1-b688-a412f5b8a4ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:57.350008 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3341b2-4ac8-47f1-b688-a412f5b8a4ae admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:57.363016 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1285/2589] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:56:57.367441 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ec9ab-4427-423d-810a-26eb5870377f None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:57.372130 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:57.379042 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ec9ab-4427-423d-810a-26eb5870377f None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:57.384404 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69d83a-e984-425d-a3ed-04855272442b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:57.385073 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69d83a-e984-425d-a3ed-04855272442b admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:57.385356 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69d83a-e984-425d-a3ed-04855272442b admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:57.385631 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69d83a-e984-425d-a3ed-04855272442b admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:57.386327 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69d83a-e984-425d-a3ed-04855272442b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:57.388266 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69d83a-e984-425d-a3ed-04855272442b admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:57.514576 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1305/2590] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:56:57 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:56:57.517128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3341b2-4ac8-47f1-b688-a412f5b8a4ae admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:57.527440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3341b2-4ac8-47f1-b688-a412f5b8a4ae admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:57.587033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562fd01-5447-401f-961b-8c805cadfa94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:57.587699 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562fd01-5447-401f-961b-8c805cadfa94 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:57.588034 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562fd01-5447-401f-961b-8c805cadfa94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:57.588342 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562fd01-5447-401f-961b-8c805cadfa94 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:57.588993 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562fd01-5447-401f-961b-8c805cadfa94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:57.591025 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562fd01-5447-401f-961b-8c805cadfa94 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:57.725204 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1286/2591] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:56:57 2022] POST /identity/v3/users => generated 291 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:56:57.728985 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69d83a-e984-425d-a3ed-04855272442b admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:57.752120 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69d83a-e984-425d-a3ed-04855272442b admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:57.777633 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763b9e4-7569-4ec8-95ea-61251d4bfea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:57.777633 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763b9e4-7569-4ec8-95ea-61251d4bfea4 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:57.777633 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763b9e4-7569-4ec8-95ea-61251d4bfea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:57.777633 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763b9e4-7569-4ec8-95ea-61251d4bfea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:57.778363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763b9e4-7569-4ec8-95ea-61251d4bfea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:57.778363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763b9e4-7569-4ec8-95ea-61251d4bfea4 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:57.792755 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1306/2592] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:56:57.796888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562fd01-5447-401f-961b-8c805cadfa94 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:57.825637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562fd01-5447-401f-961b-8c805cadfa94 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:57.837225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60c8ce-a17d-44cb-9721-131c4009ed51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:57.838174 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60c8ce-a17d-44cb-9721-131c4009ed51 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:57.839486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60c8ce-a17d-44cb-9721-131c4009ed51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:57.839486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60c8ce-a17d-44cb-9721-131c4009ed51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:57.839734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60c8ce-a17d-44cb-9721-131c4009ed51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:57.841697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60c8ce-a17d-44cb-9721-131c4009ed51 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:57.854572 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1287/2593] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:56:57.859566 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763b9e4-7569-4ec8-95ea-61251d4bfea4 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:57.871299 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763b9e4-7569-4ec8-95ea-61251d4bfea4 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:57.878034 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc6b44-72e4-4281-9f0f-ad6ec68e9aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:57.878903 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc6b44-72e4-4281-9f0f-ad6ec68e9aa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:57.879249 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc6b44-72e4-4281-9f0f-ad6ec68e9aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:57.881848 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc6b44-72e4-4281-9f0f-ad6ec68e9aa2 admin admin] PATH_INFO: `/v3/projects/611c5a90418f4cdd86c3b8e05919af9a/users/86e74d33cd6445bb872585e27643a7f4/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:57.897135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc6b44-72e4-4281-9f0f-ad6ec68e9aa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=611c5a90418f4cdd86c3b8e05919af9a, user_id=86e74d33cd6445bb872585e27643a7f4, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:57.900209 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc6b44-72e4-4281-9f0f-ad6ec68e9aa2 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:58.008205 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1307/2594] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:56:57 2022] PUT /identity/v3/projects/611c5a90418f4cdd86c3b8e05919af9a/users/86e74d33cd6445bb872585e27643a7f4/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:58.018397 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77720d4-e61d-4d1d-87d0-709cbd46cbe4 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:58.018721 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77720d4-e61d-4d1d-87d0-709cbd46cbe4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:58.019000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77720d4-e61d-4d1d-87d0-709cbd46cbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:58.048727 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f77720d4-e61d-4d1d-87d0-709cbd46cbe4 None None] MFA Rules not processed for user `86e74d33cd6445bb872585e27643a7f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:58.049986 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f77720d4-e61d-4d1d-87d0-709cbd46cbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:58.070509 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:58.143384 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1288/2595] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:56:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:58.148147 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc6b44-72e4-4281-9f0f-ad6ec68e9aa2 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:58.218999 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc6b44-72e4-4281-9f0f-ad6ec68e9aa2 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:58.223673 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f62f5-e3e8-424a-87bc-f3e4c111083a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:58.224325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f62f5-e3e8-424a-87bc-f3e4c111083a admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:58.224611 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f62f5-e3e8-424a-87bc-f3e4c111083a admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:58.224884 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f62f5-e3e8-424a-87bc-f3e4c111083a admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:58.225579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f62f5-e3e8-424a-87bc-f3e4c111083a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:58.227522 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f62f5-e3e8-424a-87bc-f3e4c111083a admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:58.374926 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77720d4-e61d-4d1d-87d0-709cbd46cbe4 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:58.376782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1308/2596] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:56:58 2022] POST /identity/v3/projects => generated 379 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:56:58.395406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77720d4-e61d-4d1d-87d0-709cbd46cbe4 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:58.466854 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07f2d8-8392-4522-bd0d-a5141eb9238f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:58.471290 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07f2d8-8392-4522-bd0d-a5141eb9238f admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:58.471290 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07f2d8-8392-4522-bd0d-a5141eb9238f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:58.471572 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07f2d8-8392-4522-bd0d-a5141eb9238f admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:58.472218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07f2d8-8392-4522-bd0d-a5141eb9238f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:58.474205 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07f2d8-8392-4522-bd0d-a5141eb9238f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:58.604215 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1289/2597] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:56:58 2022] POST /identity/v3/users => generated 296 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:56:58.608733 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f62f5-e3e8-424a-87bc-f3e4c111083a admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:58.622621 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f62f5-e3e8-424a-87bc-f3e4c111083a admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:58.633946 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587e5c3-42a8-474d-85af-c315a32698fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:58.634678 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587e5c3-42a8-474d-85af-c315a32698fd admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:58.634965 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587e5c3-42a8-474d-85af-c315a32698fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:58.635237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587e5c3-42a8-474d-85af-c315a32698fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:58.646141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5587e5c3-42a8-474d-85af-c315a32698fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:58.649144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5587e5c3-42a8-474d-85af-c315a32698fd admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:58.661700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1309/2598] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:56:58.666235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07f2d8-8392-4522-bd0d-a5141eb9238f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:58.677474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07f2d8-8392-4522-bd0d-a5141eb9238f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:58.682548 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62189a-ef74-4553-b918-ea8505d187e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:58.683765 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62189a-ef74-4553-b918-ea8505d187e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:58.684049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62189a-ef74-4553-b918-ea8505d187e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:58.684320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62189a-ef74-4553-b918-ea8505d187e6 admin admin] PATH_INFO: `/v3/projects/1ba10a7305ab4e689c0015c0ce09752c/users/2f04a10bd27b4d05be373dfe2b0a1f56/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:58.703968 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62189a-ef74-4553-b918-ea8505d187e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ba10a7305ab4e689c0015c0ce09752c, user_id=2f04a10bd27b4d05be373dfe2b0a1f56, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:58.706939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62189a-ef74-4553-b918-ea8505d187e6 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:58.782473 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1290/2599] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:56:58 2022] PUT /identity/v3/projects/1ba10a7305ab4e689c0015c0ce09752c/users/2f04a10bd27b4d05be373dfe2b0a1f56/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:58.787617 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587e5c3-42a8-474d-85af-c315a32698fd admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:58.799169 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587e5c3-42a8-474d-85af-c315a32698fd admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:58.861246 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b434b3d-a717-44c9-8174-9f41d5eddfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:58.861951 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b434b3d-a717-44c9-8174-9f41d5eddfc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:58.862194 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b434b3d-a717-44c9-8174-9f41d5eddfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:58.862471 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b434b3d-a717-44c9-8174-9f41d5eddfc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:58.863233 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b434b3d-a717-44c9-8174-9f41d5eddfc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:58.864969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b434b3d-a717-44c9-8174-9f41d5eddfc7 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:58.876055 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1310/2600] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:56:58.880258 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62189a-ef74-4553-b918-ea8505d187e6 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:58.891160 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62189a-ef74-4553-b918-ea8505d187e6 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:58.895846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4498243-d37b-45cd-a7ae-d69a30a2104f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:58.896726 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4498243-d37b-45cd-a7ae-d69a30a2104f admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:58.897008 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4498243-d37b-45cd-a7ae-d69a30a2104f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:58.897280 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4498243-d37b-45cd-a7ae-d69a30a2104f admin admin] PATH_INFO: `/v3/projects/1ba10a7305ab4e689c0015c0ce09752c/users/2f04a10bd27b4d05be373dfe2b0a1f56/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:58.901696 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4498243-d37b-45cd-a7ae-d69a30a2104f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ba10a7305ab4e689c0015c0ce09752c, user_id=2f04a10bd27b4d05be373dfe2b0a1f56, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:58.915564 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4498243-d37b-45cd-a7ae-d69a30a2104f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:58.934966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1291/2601] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:56:58 2022] PUT /identity/v3/projects/1ba10a7305ab4e689c0015c0ce09752c/users/2f04a10bd27b4d05be373dfe2b0a1f56/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:56:58.954155 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8b946-c8f6-4acf-a49c-1115374a1ecc None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:58.954712 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8b946-c8f6-4acf-a49c-1115374a1ecc None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:58.954991 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8b946-c8f6-4acf-a49c-1115374a1ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:58.996584 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-2fa8b946-c8f6-4acf-a49c-1115374a1ecc None None] MFA Rules not processed for user `2f04a10bd27b4d05be373dfe2b0a1f56`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:58.997799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-2fa8b946-c8f6-4acf-a49c-1115374a1ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:59.003537 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:59.086785 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4498243-d37b-45cd-a7ae-d69a30a2104f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:59.088915 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f4498243-d37b-45cd-a7ae-d69a30a2104f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:59.090513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1311/2602] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:56:58 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:59.103306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8b946-c8f6-4acf-a49c-1115374a1ecc None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:59.104817 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-2fa8b946-c8f6-4acf-a49c-1115374a1ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:59.160363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4498243-d37b-45cd-a7ae-d69a30a2104f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:59.166277 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d99407-0e84-49df-8c4c-9acb740a58d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:59.172071 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d99407-0e84-49df-8c4c-9acb740a58d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:59.172071 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d99407-0e84-49df-8c4c-9acb740a58d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:59.172562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8b946-c8f6-4acf-a49c-1115374a1ecc None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:59.172663 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d99407-0e84-49df-8c4c-9acb740a58d6 admin admin] PATH_INFO: `/v3/users/6cc5a4676f6a416cbdc8c67ebd80cdf3` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:59.173431 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9af71e-9c10-491c-9384-c0cbaee056df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:59.174271 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9af71e-9c10-491c-9384-c0cbaee056df service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:59.174650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9af71e-9c10-491c-9384-c0cbaee056df service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:59.174991 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9af71e-9c10-491c-9384-c0cbaee056df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:59.183745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d99407-0e84-49df-8c4c-9acb740a58d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cc5a4676f6a416cbdc8c67ebd80cdf3)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:59.185397 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9af71e-9c10-491c-9384-c0cbaee056df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:59.185714 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d99407-0e84-49df-8c4c-9acb740a58d6 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:59.188028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9af71e-9c10-491c-9384-c0cbaee056df service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:59.197916 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1312/2603] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:56:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:56:59.308479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9af71e-9c10-491c-9384-c0cbaee056df service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:59.309925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-4b9af71e-9c10-491c-9384-c0cbaee056df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:59.376649 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9af71e-9c10-491c-9384-c0cbaee056df service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:59.376649 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb74efc-4533-4a2b-b827-930edfa4a255 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:59.377134 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb74efc-4533-4a2b-b827-930edfa4a255 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:59.377760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb74efc-4533-4a2b-b827-930edfa4a255 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:59.377931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb74efc-4533-4a2b-b827-930edfa4a255 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:59.380421 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-6cb74efc-4533-4a2b-b827-930edfa4a255 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:59.393076 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb74efc-4533-4a2b-b827-930edfa4a255 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:59.394522 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb74efc-4533-4a2b-b827-930edfa4a255 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:59.405184 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1313/2604] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:56:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:56:59.410897 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-53d99407-0e84-49df-8c4c-9acb740a58d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cc5a4676f6a416cbdc8c67ebd80cdf3'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:59.420059 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-53d99407-0e84-49df-8c4c-9acb740a58d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cc5a4676f6a416cbdc8c67ebd80cdf3'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:59.421339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-53d99407-0e84-49df-8c4c-9acb740a58d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cc5a4676f6a416cbdc8c67ebd80cdf3'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:59.421943 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-53d99407-0e84-49df-8c4c-9acb740a58d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cc5a4676f6a416cbdc8c67ebd80cdf3'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:59.435933 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-53d99407-0e84-49df-8c4c-9acb740a58d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cc5a4676f6a416cbdc8c67ebd80cdf3'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:59.550192 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1292/2605] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:56:59 2022] DELETE /identity/v3/users/6cc5a4676f6a416cbdc8c67ebd80cdf3 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:59.790228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:59.832390 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb74efc-4533-4a2b-b827-930edfa4a255 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:59.848944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb74efc-4533-4a2b-b827-930edfa4a255 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:59.853129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d99407-0e84-49df-8c4c-9acb740a58d6 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:59.855245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-53d99407-0e84-49df-8c4c-9acb740a58d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:59.921372 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ad7e3-64e5-429e-9c61-d8e2d42c6507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:59.923725 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ad7e3-64e5-429e-9c61-d8e2d42c6507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:59.924182 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ad7e3-64e5-429e-9c61-d8e2d42c6507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:59.924645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ad7e3-64e5-429e-9c61-d8e2d42c6507 admin admin] PATH_INFO: `/v3/projects/2e810a9bc3b44c4aa8ca2ddd0fa68f7b` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:59.927846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d99407-0e84-49df-8c4c-9acb740a58d6 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:59.932700 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1d7dd-ce12-4bc7-9558-3998b0276597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:59.933625 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1d7dd-ce12-4bc7-9558-3998b0276597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:59.933910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1d7dd-ce12-4bc7-9558-3998b0276597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:59.934184 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1d7dd-ce12-4bc7-9558-3998b0276597 admin admin] PATH_INFO: `/v3/users/86e74d33cd6445bb872585e27643a7f4` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:59.935519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1d7dd-ce12-4bc7-9558-3998b0276597 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86e74d33cd6445bb872585e27643a7f4)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:59.937371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ad7e3-64e5-429e-9c61-d8e2d42c6507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e810a9bc3b44c4aa8ca2ddd0fa68f7b)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:59.937899 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1d7dd-ce12-4bc7-9558-3998b0276597 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:59.939338 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ad7e3-64e5-429e-9c61-d8e2d42c6507 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:59.980474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:00.147881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f02ad7e3-64e5-429e-9c61-d8e2d42c6507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e810a9bc3b44c4aa8ca2ddd0fa68f7b', 'request_id': 'req-f02ad7e3-64e5-429e-9c61-d8e2d42c6507'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:00.148175 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-94a1d7dd-ce12-4bc7-9558-3998b0276597 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86e74d33cd6445bb872585e27643a7f4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:00.155983 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-94a1d7dd-ce12-4bc7-9558-3998b0276597 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86e74d33cd6445bb872585e27643a7f4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:00.156235 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f02ad7e3-64e5-429e-9c61-d8e2d42c6507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e810a9bc3b44c4aa8ca2ddd0fa68f7b', 'request_id': 'req-f02ad7e3-64e5-429e-9c61-d8e2d42c6507'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:00.157019 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-94a1d7dd-ce12-4bc7-9558-3998b0276597 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86e74d33cd6445bb872585e27643a7f4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:00.157779 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-94a1d7dd-ce12-4bc7-9558-3998b0276597 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86e74d33cd6445bb872585e27643a7f4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:00.170127 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-94a1d7dd-ce12-4bc7-9558-3998b0276597 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86e74d33cd6445bb872585e27643a7f4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:00.230379 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f02ad7e3-64e5-429e-9c61-d8e2d42c6507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e810a9bc3b44c4aa8ca2ddd0fa68f7b', 'request_id': 'req-f02ad7e3-64e5-429e-9c61-d8e2d42c6507'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:00.242753 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f02ad7e3-64e5-429e-9c61-d8e2d42c6507 admin admin] The token cache is being invalidate because project 2e810a9bc3b44c4aa8ca2ddd0fa68f7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:00.243044 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f02ad7e3-64e5-429e-9c61-d8e2d42c6507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:00.243776 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f02ad7e3-64e5-429e-9c61-d8e2d42c6507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:00.245442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1314/2606] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:56:59 2022] DELETE /identity/v3/projects/2e810a9bc3b44c4aa8ca2ddd0fa68f7b => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:00.246257 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ad7e3-64e5-429e-9c61-d8e2d42c6507 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:00.247588 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f02ad7e3-64e5-429e-9c61-d8e2d42c6507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:00.293715 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1293/2607] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:56:59 2022] DELETE /identity/v3/users/86e74d33cd6445bb872585e27643a7f4 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:00.294837 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1d7dd-ce12-4bc7-9558-3998b0276597 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:00.297051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-94a1d7dd-ce12-4bc7-9558-3998b0276597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:00.310729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ad7e3-64e5-429e-9c61-d8e2d42c6507 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:00.363088 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1d7dd-ce12-4bc7-9558-3998b0276597 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:00.367256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2660d8b-6b8d-4df2-bbde-e651779d8821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:00.367609 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b09b3-2736-452b-9348-4c3390a1e131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:00.368123 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2660d8b-6b8d-4df2-bbde-e651779d8821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:00.368404 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2660d8b-6b8d-4df2-bbde-e651779d8821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:00.368519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b09b3-2736-452b-9348-4c3390a1e131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:00.368678 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2660d8b-6b8d-4df2-bbde-e651779d8821 admin admin] PATH_INFO: `/v3/users/9040cb92bc354fbd985b40db56894042` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:00.368778 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b09b3-2736-452b-9348-4c3390a1e131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:00.368998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b09b3-2736-452b-9348-4c3390a1e131 admin admin] PATH_INFO: `/v3/users/c8c82a19ef734e058ee04f21881b7f4b` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:00.369890 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2660d8b-6b8d-4df2-bbde-e651779d8821 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9040cb92bc354fbd985b40db56894042)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:00.371925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2660d8b-6b8d-4df2-bbde-e651779d8821 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:00.385689 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b09b3-2736-452b-9348-4c3390a1e131 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8c82a19ef734e058ee04f21881b7f4b)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:00.387190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b09b3-2736-452b-9348-4c3390a1e131 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:00.396184 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:00.574463 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-a2660d8b-6b8d-4df2-bbde-e651779d8821 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9040cb92bc354fbd985b40db56894042'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:00.579238 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-464b09b3-2736-452b-9348-4c3390a1e131 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8c82a19ef734e058ee04f21881b7f4b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:00.583772 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-a2660d8b-6b8d-4df2-bbde-e651779d8821 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9040cb92bc354fbd985b40db56894042'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:00.584689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-a2660d8b-6b8d-4df2-bbde-e651779d8821 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9040cb92bc354fbd985b40db56894042'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:00.585729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-a2660d8b-6b8d-4df2-bbde-e651779d8821 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9040cb92bc354fbd985b40db56894042'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:00.587814 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-464b09b3-2736-452b-9348-4c3390a1e131 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8c82a19ef734e058ee04f21881b7f4b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:00.588823 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-464b09b3-2736-452b-9348-4c3390a1e131 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8c82a19ef734e058ee04f21881b7f4b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:00.589555 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-464b09b3-2736-452b-9348-4c3390a1e131 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8c82a19ef734e058ee04f21881b7f4b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:00.600158 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-a2660d8b-6b8d-4df2-bbde-e651779d8821 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9040cb92bc354fbd985b40db56894042'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:00.604674 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-464b09b3-2736-452b-9348-4c3390a1e131 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8c82a19ef734e058ee04f21881b7f4b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:00.670847 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1315/2608] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:00 2022] DELETE /identity/v3/users/9040cb92bc354fbd985b40db56894042 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:00.672492 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53cd0e-ed6e-4a62-8a5e-57ad60f6f342 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:00.674028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53cd0e-ed6e-4a62-8a5e-57ad60f6f342 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:00.674028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53cd0e-ed6e-4a62-8a5e-57ad60f6f342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:00.677018 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:00.691311 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-eb53cd0e-ed6e-4a62-8a5e-57ad60f6f342 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:00.692444 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-eb53cd0e-ed6e-4a62-8a5e-57ad60f6f342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:00.735254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1294/2609] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:00 2022] DELETE /identity/v3/users/c8c82a19ef734e058ee04f21881b7f4b => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:00.736887 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d919bf2-a761-455c-b592-6e2645d0cce8 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:00.737244 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d919bf2-a761-455c-b592-6e2645d0cce8 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:00.737520 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d919bf2-a761-455c-b592-6e2645d0cce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:00.757292 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-9d919bf2-a761-455c-b592-6e2645d0cce8 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:00.759015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9d919bf2-a761-455c-b592-6e2645d0cce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:00.770668 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1316/2610] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:00.789234 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb53cd0e-ed6e-4a62-8a5e-57ad60f6f342 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:00.794715 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-eb53cd0e-ed6e-4a62-8a5e-57ad60f6f342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:00.826027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1295/2611] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:00.846922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d919bf2-a761-455c-b592-6e2645d0cce8 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:00.848431 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9d919bf2-a761-455c-b592-6e2645d0cce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:00.880926 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb53cd0e-ed6e-4a62-8a5e-57ad60f6f342 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:00.919078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d919bf2-a761-455c-b592-6e2645d0cce8 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:00.926065 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00f0e6-7b40-4369-bd7e-da2967895d6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:00.927011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00f0e6-7b40-4369-bd7e-da2967895d6c service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:00.927396 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00f0e6-7b40-4369-bd7e-da2967895d6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:00.927756 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00f0e6-7b40-4369-bd7e-da2967895d6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:00.941091 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00f0e6-7b40-4369-bd7e-da2967895d6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:00.942065 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00f0e6-7b40-4369-bd7e-da2967895d6c service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:00.962777 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca06bf-40cf-4f07-956e-8f6daf8e827b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:00.964142 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca06bf-40cf-4f07-956e-8f6daf8e827b service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:00.964731 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1296/2612] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:57:00.964871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca06bf-40cf-4f07-956e-8f6daf8e827b service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:00.965143 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca06bf-40cf-4f07-956e-8f6daf8e827b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:00.977355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca06bf-40cf-4f07-956e-8f6daf8e827b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:00.979048 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca06bf-40cf-4f07-956e-8f6daf8e827b service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:00.994449 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1317/2613] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:01.066573 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00f0e6-7b40-4369-bd7e-da2967895d6c service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:01.068527 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bc00f0e6-7b40-4369-bd7e-da2967895d6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:01.086649 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00f0e6-7b40-4369-bd7e-da2967895d6c service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:01.093803 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa497b40-a3de-4aed-b780-c4bc7082f73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:01.095049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa497b40-a3de-4aed-b780-c4bc7082f73a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:01.095480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa497b40-a3de-4aed-b780-c4bc7082f73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:01.096173 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa497b40-a3de-4aed-b780-c4bc7082f73a admin admin] PATH_INFO: `/v3/projects/3827d72899c84273882f4b69098ae7b8` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:01.111466 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa497b40-a3de-4aed-b780-c4bc7082f73a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3827d72899c84273882f4b69098ae7b8)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:01.114216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa497b40-a3de-4aed-b780-c4bc7082f73a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:01.245842 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-aa497b40-a3de-4aed-b780-c4bc7082f73a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3827d72899c84273882f4b69098ae7b8', 'request_id': 'req-aa497b40-a3de-4aed-b780-c4bc7082f73a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:01.255449 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-aa497b40-a3de-4aed-b780-c4bc7082f73a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3827d72899c84273882f4b69098ae7b8', 'request_id': 'req-aa497b40-a3de-4aed-b780-c4bc7082f73a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:01.259680 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:01.351026 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-aa497b40-a3de-4aed-b780-c4bc7082f73a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3827d72899c84273882f4b69098ae7b8', 'request_id': 'req-aa497b40-a3de-4aed-b780-c4bc7082f73a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:01.361974 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:01.366731 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-aa497b40-a3de-4aed-b780-c4bc7082f73a admin admin] The token cache is being invalidate because project 3827d72899c84273882f4b69098ae7b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:01.366942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-aa497b40-a3de-4aed-b780-c4bc7082f73a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:01.367880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-aa497b40-a3de-4aed-b780-c4bc7082f73a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:01.370046 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1297/2614] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:01 2022] DELETE /identity/v3/projects/3827d72899c84273882f4b69098ae7b8 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:02.061296 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca06bf-40cf-4f07-956e-8f6daf8e827b service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:02.062283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa497b40-a3de-4aed-b780-c4bc7082f73a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:02.062883 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-12ca06bf-40cf-4f07-956e-8f6daf8e827b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:02.063057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-aa497b40-a3de-4aed-b780-c4bc7082f73a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:02.076146 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca06bf-40cf-4f07-956e-8f6daf8e827b service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:02.166422 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2d6dd-9632-4514-8bff-989e329fb8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:02.167371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2d6dd-9632-4514-8bff-989e329fb8c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:02.167658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2d6dd-9632-4514-8bff-989e329fb8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:02.167936 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2d6dd-9632-4514-8bff-989e329fb8c9 admin admin] PATH_INFO: `/v3/projects/a8df00f506154d1ca1d9da3e28b56712` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:02.169347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb2d6dd-9632-4514-8bff-989e329fb8c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8df00f506154d1ca1d9da3e28b56712)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:02.172184 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb2d6dd-9632-4514-8bff-989e329fb8c9 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:02.221474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa497b40-a3de-4aed-b780-c4bc7082f73a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:02.227360 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f1447-2581-4fbf-b96f-02359a47be86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:02.228425 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f1447-2581-4fbf-b96f-02359a47be86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:02.228882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f1447-2581-4fbf-b96f-02359a47be86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:02.229201 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f1447-2581-4fbf-b96f-02359a47be86 admin admin] PATH_INFO: `/v3/projects/611c5a90418f4cdd86c3b8e05919af9a` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:02.231577 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f1447-2581-4fbf-b96f-02359a47be86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=611c5a90418f4cdd86c3b8e05919af9a)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:02.235752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f1447-2581-4fbf-b96f-02359a47be86 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:02.314905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-5bb2d6dd-9632-4514-8bff-989e329fb8c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8df00f506154d1ca1d9da3e28b56712', 'request_id': 'req-5bb2d6dd-9632-4514-8bff-989e329fb8c9'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:02.323821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-5bb2d6dd-9632-4514-8bff-989e329fb8c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8df00f506154d1ca1d9da3e28b56712', 'request_id': 'req-5bb2d6dd-9632-4514-8bff-989e329fb8c9'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:02.352786 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:02.368653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-222f1447-2581-4fbf-b96f-02359a47be86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '611c5a90418f4cdd86c3b8e05919af9a', 'request_id': 'req-222f1447-2581-4fbf-b96f-02359a47be86'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:02.375152 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-222f1447-2581-4fbf-b96f-02359a47be86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '611c5a90418f4cdd86c3b8e05919af9a', 'request_id': 'req-222f1447-2581-4fbf-b96f-02359a47be86'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:02.402217 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-5bb2d6dd-9632-4514-8bff-989e329fb8c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8df00f506154d1ca1d9da3e28b56712', 'request_id': 'req-5bb2d6dd-9632-4514-8bff-989e329fb8c9'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:02.421991 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-5bb2d6dd-9632-4514-8bff-989e329fb8c9 admin admin] The token cache is being invalidate because project a8df00f506154d1ca1d9da3e28b56712 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:02.422287 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-5bb2d6dd-9632-4514-8bff-989e329fb8c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:02.422974 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-5bb2d6dd-9632-4514-8bff-989e329fb8c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:02.424905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1318/2615] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:02 2022] DELETE /identity/v3/projects/a8df00f506154d1ca1d9da3e28b56712 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:02.429513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f6945-8ec0-4174-8f14-1afdd9794a63 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:02.429808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f6945-8ec0-4174-8f14-1afdd9794a63 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:02.430221 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f6945-8ec0-4174-8f14-1afdd9794a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:02.448125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-a79f6945-8ec0-4174-8f14-1afdd9794a63 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:02.449367 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a79f6945-8ec0-4174-8f14-1afdd9794a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:02.466052 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-222f1447-2581-4fbf-b96f-02359a47be86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '611c5a90418f4cdd86c3b8e05919af9a', 'request_id': 'req-222f1447-2581-4fbf-b96f-02359a47be86'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:02.479341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-222f1447-2581-4fbf-b96f-02359a47be86 admin admin] The token cache is being invalidate because project 611c5a90418f4cdd86c3b8e05919af9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:02.480197 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-222f1447-2581-4fbf-b96f-02359a47be86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:02.480483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-222f1447-2581-4fbf-b96f-02359a47be86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:02.484833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1298/2616] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:02 2022] DELETE /identity/v3/projects/611c5a90418f4cdd86c3b8e05919af9a => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:02.492004 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f1447-2581-4fbf-b96f-02359a47be86 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:02.492004 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-222f1447-2581-4fbf-b96f-02359a47be86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:02.536044 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1319/2617] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:02.546803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f363935-64de-458b-b8fa-4eaaefc0c7db None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:02.546803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f363935-64de-458b-b8fa-4eaaefc0c7db None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:02.547018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f363935-64de-458b-b8fa-4eaaefc0c7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:02.571316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-6f363935-64de-458b-b8fa-4eaaefc0c7db None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:02.572744 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-6f363935-64de-458b-b8fa-4eaaefc0c7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:02.574072 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f1447-2581-4fbf-b96f-02359a47be86 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:02.581268 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfe570-0776-4a8d-8780-cf6ca2acbaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:02.582695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfe570-0776-4a8d-8780-cf6ca2acbaba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:02.582695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfe570-0776-4a8d-8780-cf6ca2acbaba admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:02.582932 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfe570-0776-4a8d-8780-cf6ca2acbaba admin admin] PATH_INFO: `/v3/users/2f04a10bd27b4d05be373dfe2b0a1f56` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:02.584361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfe570-0776-4a8d-8780-cf6ca2acbaba admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f04a10bd27b4d05be373dfe2b0a1f56)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:02.586642 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfe570-0776-4a8d-8780-cf6ca2acbaba admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:02.587628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1320/2618] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:02.591920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f363935-64de-458b-b8fa-4eaaefc0c7db None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:02.605798 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f363935-64de-458b-b8fa-4eaaefc0c7db None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:02.613904 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99573d13-e515-4add-b367-1a830d1c675a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:02.614879 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99573d13-e515-4add-b367-1a830d1c675a admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:02.615332 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99573d13-e515-4add-b367-1a830d1c675a admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:02.615886 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99573d13-e515-4add-b367-1a830d1c675a admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:02.616784 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99573d13-e515-4add-b367-1a830d1c675a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:57:02.617767 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99573d13-e515-4add-b367-1a830d1c675a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:02.619504 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99573d13-e515-4add-b367-1a830d1c675a admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:02.639125 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1321/2619] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:57:02.650617 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99573d13-e515-4add-b367-1a830d1c675a admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:02.657710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:02.662017 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99573d13-e515-4add-b367-1a830d1c675a admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:02.666989 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e87ca-ec01-4393-9956-fccf85d2b828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:02.667691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e87ca-ec01-4393-9956-fccf85d2b828 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:02.668024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e87ca-ec01-4393-9956-fccf85d2b828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:02.668259 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e87ca-ec01-4393-9956-fccf85d2b828 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:02.668980 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e87ca-ec01-4393-9956-fccf85d2b828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:02.671235 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e87ca-ec01-4393-9956-fccf85d2b828 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:02.809617 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1322/2620] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:02 2022] POST /identity/v3/projects => generated 371 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:02.814796 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e87ca-ec01-4393-9956-fccf85d2b828 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:02.841655 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3ebfe570-0776-4a8d-8780-cf6ca2acbaba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f04a10bd27b4d05be373dfe2b0a1f56'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:02.849162 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3ebfe570-0776-4a8d-8780-cf6ca2acbaba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f04a10bd27b4d05be373dfe2b0a1f56'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:02.849953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3ebfe570-0776-4a8d-8780-cf6ca2acbaba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f04a10bd27b4d05be373dfe2b0a1f56'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:02.850740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3ebfe570-0776-4a8d-8780-cf6ca2acbaba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f04a10bd27b4d05be373dfe2b0a1f56'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:02.875343 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3ebfe570-0776-4a8d-8780-cf6ca2acbaba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f04a10bd27b4d05be373dfe2b0a1f56'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:02.888954 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e87ca-ec01-4393-9956-fccf85d2b828 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:02.896129 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684d707-0236-4137-9e48-1cc6f456f008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:02.897026 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684d707-0236-4137-9e48-1cc6f456f008 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:02.897346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684d707-0236-4137-9e48-1cc6f456f008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:02.897625 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684d707-0236-4137-9e48-1cc6f456f008 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:02.898332 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684d707-0236-4137-9e48-1cc6f456f008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:02.900491 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684d707-0236-4137-9e48-1cc6f456f008 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:03.001172 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1299/2621] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:02 2022] DELETE /identity/v3/users/2f04a10bd27b4d05be373dfe2b0a1f56 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:03.001759 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1323/2622] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:02 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:03.012697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfe570-0776-4a8d-8780-cf6ca2acbaba admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:03.024375 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3ebfe570-0776-4a8d-8780-cf6ca2acbaba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:03.100218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:03.145331 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfe570-0776-4a8d-8780-cf6ca2acbaba admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:03.156548 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890e8bd-1dd2-4f63-a90b-66fec36d8c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:03.157746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890e8bd-1dd2-4f63-a90b-66fec36d8c81 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:03.157904 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890e8bd-1dd2-4f63-a90b-66fec36d8c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:03.158195 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890e8bd-1dd2-4f63-a90b-66fec36d8c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:03.158974 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890e8bd-1dd2-4f63-a90b-66fec36d8c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:03.160874 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890e8bd-1dd2-4f63-a90b-66fec36d8c81 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:03.172536 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1300/2623] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:03.176490 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684d707-0236-4137-9e48-1cc6f456f008 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:03.187659 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684d707-0236-4137-9e48-1cc6f456f008 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:03.193563 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed62136-eb89-4b4e-a063-3286c2f8f7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:03.194363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed62136-eb89-4b4e-a063-3286c2f8f7fd admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:03.194789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed62136-eb89-4b4e-a063-3286c2f8f7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:03.195141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed62136-eb89-4b4e-a063-3286c2f8f7fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:03.196215 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed62136-eb89-4b4e-a063-3286c2f8f7fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:03.197665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed62136-eb89-4b4e-a063-3286c2f8f7fd admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:03.208863 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1324/2624] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:03.219275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890e8bd-1dd2-4f63-a90b-66fec36d8c81 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:03.230221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890e8bd-1dd2-4f63-a90b-66fec36d8c81 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:03.237279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f61d5-c37d-4733-8850-a7bcc023f735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:03.238176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f61d5-c37d-4733-8850-a7bcc023f735 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:03.239012 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f61d5-c37d-4733-8850-a7bcc023f735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:03.239365 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f61d5-c37d-4733-8850-a7bcc023f735 admin admin] PATH_INFO: `/v3/projects/a00e3dc3a73a42bdb9426b0213933056/users/69c345b278e4437d98f9b1f412bbca49/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:03.275808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f61d5-c37d-4733-8850-a7bcc023f735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a00e3dc3a73a42bdb9426b0213933056, user_id=69c345b278e4437d98f9b1f412bbca49, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:03.280942 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f61d5-c37d-4733-8850-a7bcc023f735 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:03.386991 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fd6ab-95f4-4b20-8f73-2e14c1b06341 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:03.387345 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1301/2625] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:03 2022] PUT /identity/v3/projects/a00e3dc3a73a42bdb9426b0213933056/users/69c345b278e4437d98f9b1f412bbca49/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:03.387581 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fd6ab-95f4-4b20-8f73-2e14c1b06341 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:03.387954 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fd6ab-95f4-4b20-8f73-2e14c1b06341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:03.413810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-e22fd6ab-95f4-4b20-8f73-2e14c1b06341 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:03.415083 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e22fd6ab-95f4-4b20-8f73-2e14c1b06341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:03.481569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f61d5-c37d-4733-8850-a7bcc023f735 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:03.482671 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1325/2626] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:03.485255 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-df1f61d5-c37d-4733-8850-a7bcc023f735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:03.496476 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22fd6ab-95f4-4b20-8f73-2e14c1b06341 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:03.498114 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e22fd6ab-95f4-4b20-8f73-2e14c1b06341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:03.517515 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f61d5-c37d-4733-8850-a7bcc023f735 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:03.529441 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead91019-6a90-43b9-bfee-9f96515b59d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:03.533752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead91019-6a90-43b9-bfee-9f96515b59d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:03.533954 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead91019-6a90-43b9-bfee-9f96515b59d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:03.534202 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead91019-6a90-43b9-bfee-9f96515b59d5 admin admin] PATH_INFO: `/v3/projects/1ba10a7305ab4e689c0015c0ce09752c` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:03.536314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead91019-6a90-43b9-bfee-9f96515b59d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ba10a7305ab4e689c0015c0ce09752c)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:03.544552 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead91019-6a90-43b9-bfee-9f96515b59d5 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:03.571975 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22fd6ab-95f4-4b20-8f73-2e14c1b06341 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:03.575864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52349b9b-8967-4134-831f-5bf73bc60178 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:03.576534 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52349b9b-8967-4134-831f-5bf73bc60178 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:03.576817 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52349b9b-8967-4134-831f-5bf73bc60178 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:03.577313 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52349b9b-8967-4134-831f-5bf73bc60178 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:03.586408 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52349b9b-8967-4134-831f-5bf73bc60178 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:03.587878 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52349b9b-8967-4134-831f-5bf73bc60178 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:03.598215 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1326/2627] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:57:03.627045 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52349b9b-8967-4134-831f-5bf73bc60178 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:03.628566 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-52349b9b-8967-4134-831f-5bf73bc60178 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:03.638382 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:03.643195 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52349b9b-8967-4134-831f-5bf73bc60178 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:03.658936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ead91019-6a90-43b9-bfee-9f96515b59d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ba10a7305ab4e689c0015c0ce09752c', 'request_id': 'req-ead91019-6a90-43b9-bfee-9f96515b59d5'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:03.665338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ead91019-6a90-43b9-bfee-9f96515b59d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ba10a7305ab4e689c0015c0ce09752c', 'request_id': 'req-ead91019-6a90-43b9-bfee-9f96515b59d5'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:03.700041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655980b0-07a3-4578-a6ed-e660c0c96446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:03.700908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655980b0-07a3-4578-a6ed-e660c0c96446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:03.701190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655980b0-07a3-4578-a6ed-e660c0c96446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:03.701464 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655980b0-07a3-4578-a6ed-e660c0c96446 admin admin] PATH_INFO: `/v3/users/41cb2b9a5156470aaeca9cc10f402ef9` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:03.702786 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655980b0-07a3-4578-a6ed-e660c0c96446 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41cb2b9a5156470aaeca9cc10f402ef9)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:03.704702 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655980b0-07a3-4578-a6ed-e660c0c96446 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:03.750339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ead91019-6a90-43b9-bfee-9f96515b59d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ba10a7305ab4e689c0015c0ce09752c', 'request_id': 'req-ead91019-6a90-43b9-bfee-9f96515b59d5'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:03.761750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ead91019-6a90-43b9-bfee-9f96515b59d5 admin admin] The token cache is being invalidate because project 1ba10a7305ab4e689c0015c0ce09752c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:03.762035 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ead91019-6a90-43b9-bfee-9f96515b59d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:03.762716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ead91019-6a90-43b9-bfee-9f96515b59d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:03.764274 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1302/2628] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:03 2022] DELETE /identity/v3/projects/1ba10a7305ab4e689c0015c0ce09752c => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:03.769381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e649067-f434-4d73-838b-5a1e7db97596 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:03.769671 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e649067-f434-4d73-838b-5a1e7db97596 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:03.769949 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e649067-f434-4d73-838b-5a1e7db97596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:03.788619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-8e649067-f434-4d73-838b-5a1e7db97596 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:03.789829 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-8e649067-f434-4d73-838b-5a1e7db97596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:03.801495 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1303/2629] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:03.809537 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6498143-3231-417a-99bd-e1c9c4a4109f None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:03.811283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6498143-3231-417a-99bd-e1c9c4a4109f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:03.811283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6498143-3231-417a-99bd-e1c9c4a4109f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:03.829872 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f6498143-3231-417a-99bd-e1c9c4a4109f None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:03.830674 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f6498143-3231-417a-99bd-e1c9c4a4109f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:03.839021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1304/2630] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:03.843711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6498143-3231-417a-99bd-e1c9c4a4109f None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:03.853060 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6498143-3231-417a-99bd-e1c9c4a4109f None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:03.858026 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8f8d0-f810-4cbd-a011-f24e10e72447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:03.858871 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8f8d0-f810-4cbd-a011-f24e10e72447 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:03.859158 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8f8d0-f810-4cbd-a011-f24e10e72447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:03.859440 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8f8d0-f810-4cbd-a011-f24e10e72447 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:03.860165 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8f8d0-f810-4cbd-a011-f24e10e72447 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:57:03.860476 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8f8d0-f810-4cbd-a011-f24e10e72447 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:03.862080 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8f8d0-f810-4cbd-a011-f24e10e72447 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:03.873678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1305/2631] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:57:03.877941 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8f8d0-f810-4cbd-a011-f24e10e72447 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:03.887654 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8f8d0-f810-4cbd-a011-f24e10e72447 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:03.893270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977398b-3020-45e4-bb52-8d7c71fed23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:03.893922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977398b-3020-45e4-bb52-8d7c71fed23f admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:03.894206 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977398b-3020-45e4-bb52-8d7c71fed23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:03.894508 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977398b-3020-45e4-bb52-8d7c71fed23f admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:03.895208 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977398b-3020-45e4-bb52-8d7c71fed23f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:03.897079 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977398b-3020-45e4-bb52-8d7c71fed23f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:03.922620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-655980b0-07a3-4578-a6ed-e660c0c96446 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41cb2b9a5156470aaeca9cc10f402ef9'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:03.929857 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-655980b0-07a3-4578-a6ed-e660c0c96446 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41cb2b9a5156470aaeca9cc10f402ef9'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:03.930620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-655980b0-07a3-4578-a6ed-e660c0c96446 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41cb2b9a5156470aaeca9cc10f402ef9'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:03.931242 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-655980b0-07a3-4578-a6ed-e660c0c96446 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41cb2b9a5156470aaeca9cc10f402ef9'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:03.944138 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-655980b0-07a3-4578-a6ed-e660c0c96446 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41cb2b9a5156470aaeca9cc10f402ef9'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:04.008497 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1306/2632] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:03 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:04.009430 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977398b-3020-45e4-bb52-8d7c71fed23f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:04.011129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5977398b-3020-45e4-bb52-8d7c71fed23f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:04.024650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1327/2633] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:03 2022] DELETE /identity/v3/users/41cb2b9a5156470aaeca9cc10f402ef9 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:04.066504 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:04.087865 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977398b-3020-45e4-bb52-8d7c71fed23f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:04.147269 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f99c6-09a5-4920-a497-b07bc3e199b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:04.148429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f99c6-09a5-4920-a497-b07bc3e199b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:04.148722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f99c6-09a5-4920-a497-b07bc3e199b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:04.148998 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f99c6-09a5-4920-a497-b07bc3e199b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:04.149695 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f99c6-09a5-4920-a497-b07bc3e199b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:04.151667 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f99c6-09a5-4920-a497-b07bc3e199b7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:04.255861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1307/2634] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:04 2022] POST /identity/v3/users => generated 292 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:04.260435 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655980b0-07a3-4578-a6ed-e660c0c96446 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:04.273875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655980b0-07a3-4578-a6ed-e660c0c96446 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:04.278860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0ce36-261b-446c-a442-84ff714c1d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:04.279629 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0ce36-261b-446c-a442-84ff714c1d57 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:04.280046 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0ce36-261b-446c-a442-84ff714c1d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:04.280368 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0ce36-261b-446c-a442-84ff714c1d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:04.281221 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f0ce36-261b-446c-a442-84ff714c1d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:04.282957 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f0ce36-261b-446c-a442-84ff714c1d57 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:04.294194 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1328/2635] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:04.298682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f99c6-09a5-4920-a497-b07bc3e199b7 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:04.309318 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f99c6-09a5-4920-a497-b07bc3e199b7 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:04.314747 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dbc6ff-f8dc-4236-ba0f-06a25fddbf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:04.315511 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbc6ff-f8dc-4236-ba0f-06a25fddbf6f admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:04.315864 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbc6ff-f8dc-4236-ba0f-06a25fddbf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:04.316148 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbc6ff-f8dc-4236-ba0f-06a25fddbf6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:04.317164 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbc6ff-f8dc-4236-ba0f-06a25fddbf6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:04.318818 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbc6ff-f8dc-4236-ba0f-06a25fddbf6f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:04.331054 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1308/2636] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:04.335448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0ce36-261b-446c-a442-84ff714c1d57 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:04.346149 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0ce36-261b-446c-a442-84ff714c1d57 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:04.350863 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8f1ee-9a5c-42d9-ae9d-817d1ba9ed9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:04.359181 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8f1ee-9a5c-42d9-ae9d-817d1ba9ed9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:04.359181 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8f1ee-9a5c-42d9-ae9d-817d1ba9ed9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:04.359181 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8f1ee-9a5c-42d9-ae9d-817d1ba9ed9f admin admin] PATH_INFO: `/v3/projects/cd4d22efb11c4436a4fdbcf0b01a3834/users/e5db0e1d7dfb4f8d9aff3614a5d6a06a/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:04.368844 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8f1ee-9a5c-42d9-ae9d-817d1ba9ed9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd4d22efb11c4436a4fdbcf0b01a3834, user_id=e5db0e1d7dfb4f8d9aff3614a5d6a06a, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:04.371670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8f1ee-9a5c-42d9-ae9d-817d1ba9ed9f admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:04.474352 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1329/2637] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:04 2022] PUT /identity/v3/projects/cd4d22efb11c4436a4fdbcf0b01a3834/users/e5db0e1d7dfb4f8d9aff3614a5d6a06a/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:04.482214 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f18b21-95c0-4c40-8755-4573f69ee142 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:04.482514 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f18b21-95c0-4c40-8755-4573f69ee142 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:04.482834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f18b21-95c0-4c40-8755-4573f69ee142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:04.520429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-75f18b21-95c0-4c40-8755-4573f69ee142 None None] MFA Rules not processed for user `e5db0e1d7dfb4f8d9aff3614a5d6a06a`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:04.521653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-75f18b21-95c0-4c40-8755-4573f69ee142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:04.586402 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1309/2638] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:04.591001 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8f1ee-9a5c-42d9-ae9d-817d1ba9ed9f admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:04.595078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f18b21-95c0-4c40-8755-4573f69ee142 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:04.596912 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-75f18b21-95c0-4c40-8755-4573f69ee142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:04.651945 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:04.671686 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8f1ee-9a5c-42d9-ae9d-817d1ba9ed9f admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:04.678219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed076697-e2d8-4f4c-87fe-5756e43120ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:04.679143 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed076697-e2d8-4f4c-87fe-5756e43120ba admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:04.679562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed076697-e2d8-4f4c-87fe-5756e43120ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:04.680199 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed076697-e2d8-4f4c-87fe-5756e43120ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:04.681229 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed076697-e2d8-4f4c-87fe-5756e43120ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:04.683897 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed076697-e2d8-4f4c-87fe-5756e43120ba admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:04.688540 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:04.693190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f18b21-95c0-4c40-8755-4573f69ee142 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:04.699856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9337e-5552-43c5-bfb2-c6efff4c3986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:04.700899 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9337e-5552-43c5-bfb2-c6efff4c3986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:04.701156 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9337e-5552-43c5-bfb2-c6efff4c3986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:04.701478 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9337e-5552-43c5-bfb2-c6efff4c3986 admin admin] PATH_INFO: `/v3/projects/4c0275f5304e4d9eae19c46a4f161217` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:04.702800 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9337e-5552-43c5-bfb2-c6efff4c3986 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c0275f5304e4d9eae19c46a4f161217)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:04.704953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9337e-5552-43c5-bfb2-c6efff4c3986 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:04.999081 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1330/2639] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:04 2022] POST /identity/v3/projects => generated 381 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:05.002986 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed076697-e2d8-4f4c-87fe-5756e43120ba admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:05.034033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a3c9337e-5552-43c5-bfb2-c6efff4c3986 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c0275f5304e4d9eae19c46a4f161217', 'request_id': 'req-a3c9337e-5552-43c5-bfb2-c6efff4c3986'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:05.045623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a3c9337e-5552-43c5-bfb2-c6efff4c3986 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c0275f5304e4d9eae19c46a4f161217', 'request_id': 'req-a3c9337e-5552-43c5-bfb2-c6efff4c3986'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:05.075603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed076697-e2d8-4f4c-87fe-5756e43120ba admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:05.081647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019ee78-5068-4b64-b905-46e38d6b299d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:05.082620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019ee78-5068-4b64-b905-46e38d6b299d admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:05.082620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019ee78-5068-4b64-b905-46e38d6b299d admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:05.082883 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019ee78-5068-4b64-b905-46e38d6b299d admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:05.083572 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019ee78-5068-4b64-b905-46e38d6b299d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:05.085489 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019ee78-5068-4b64-b905-46e38d6b299d admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:05.149190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a3c9337e-5552-43c5-bfb2-c6efff4c3986 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c0275f5304e4d9eae19c46a4f161217', 'request_id': 'req-a3c9337e-5552-43c5-bfb2-c6efff4c3986'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:05.160794 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a3c9337e-5552-43c5-bfb2-c6efff4c3986 admin admin] The token cache is being invalidate because project 4c0275f5304e4d9eae19c46a4f161217 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:05.161080 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a3c9337e-5552-43c5-bfb2-c6efff4c3986 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:05.161705 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a3c9337e-5552-43c5-bfb2-c6efff4c3986 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:05.163270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1310/2640] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:04 2022] DELETE /identity/v3/projects/4c0275f5304e4d9eae19c46a4f161217 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:05.167844 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236ad0f-f72d-470c-8c56-370e0fe45f3e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:05.168128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236ad0f-f72d-470c-8c56-370e0fe45f3e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:05.168400 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236ad0f-f72d-470c-8c56-370e0fe45f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:05.181213 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1331/2641] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:05 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:05.185171 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019ee78-5068-4b64-b905-46e38d6b299d admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:05.186962 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-e019ee78-5068-4b64-b905-46e38d6b299d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:05.188728 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a236ad0f-f72d-470c-8c56-370e0fe45f3e None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:05.189902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a236ad0f-f72d-470c-8c56-370e0fe45f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:05.198026 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1311/2642] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:05.199598 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019ee78-5068-4b64-b905-46e38d6b299d admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:05.204257 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59562db7-c12f-4cd1-beb5-2d2243426fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:05.204915 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59562db7-c12f-4cd1-beb5-2d2243426fbe admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:05.205204 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59562db7-c12f-4cd1-beb5-2d2243426fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:05.205497 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59562db7-c12f-4cd1-beb5-2d2243426fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:05.206236 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59562db7-c12f-4cd1-beb5-2d2243426fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:05.207864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59562db7-c12f-4cd1-beb5-2d2243426fbe admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:05.210626 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ed92e-ad72-45b2-a939-a10263d7091d None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:05.210921 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ed92e-ad72-45b2-a939-a10263d7091d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:05.211229 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ed92e-ad72-45b2-a939-a10263d7091d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:05.221369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1332/2643] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:05.222297 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59562db7-c12f-4cd1-beb5-2d2243426fbe admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:05.230133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-947ed92e-ad72-45b2-a939-a10263d7091d None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:05.232874 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-947ed92e-ad72-45b2-a939-a10263d7091d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:05.232998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59562db7-c12f-4cd1-beb5-2d2243426fbe admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:05.238049 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4347f-79f8-4425-a55c-61444911ab3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:05.238998 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4347f-79f8-4425-a55c-61444911ab3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:05.239285 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4347f-79f8-4425-a55c-61444911ab3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:05.239564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4347f-79f8-4425-a55c-61444911ab3c admin admin] PATH_INFO: `/v3/projects/8e549e85424745129903cd1ed262b997/users/db8febf735bd449eabafadcf14440ff7/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:05.240877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1312/2644] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:05.245033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ed92e-ad72-45b2-a939-a10263d7091d None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:05.254813 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ed92e-ad72-45b2-a939-a10263d7091d None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:05.255860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c4347f-79f8-4425-a55c-61444911ab3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e549e85424745129903cd1ed262b997, user_id=db8febf735bd449eabafadcf14440ff7, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:05.258720 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c4347f-79f8-4425-a55c-61444911ab3c admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:05.260568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43c274-5e7b-42e0-b680-61b8588849ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:05.261305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43c274-5e7b-42e0-b680-61b8588849ff admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:05.261659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43c274-5e7b-42e0-b680-61b8588849ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:05.262283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43c274-5e7b-42e0-b680-61b8588849ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:05.262978 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43c274-5e7b-42e0-b680-61b8588849ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:57:05.263355 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43c274-5e7b-42e0-b680-61b8588849ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:05.266027 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43c274-5e7b-42e0-b680-61b8588849ff admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:05.279207 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1313/2645] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:57:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:57:05.284052 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43c274-5e7b-42e0-b680-61b8588849ff admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:05.294556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43c274-5e7b-42e0-b680-61b8588849ff admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:05.299495 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45122d2b-1586-452e-8e52-bf4fc17a5751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:05.300154 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45122d2b-1586-452e-8e52-bf4fc17a5751 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:05.300444 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45122d2b-1586-452e-8e52-bf4fc17a5751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:05.300724 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45122d2b-1586-452e-8e52-bf4fc17a5751 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:05.301420 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45122d2b-1586-452e-8e52-bf4fc17a5751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:05.303434 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45122d2b-1586-452e-8e52-bf4fc17a5751 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:05.350871 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1333/2646] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:05 2022] PUT /identity/v3/projects/8e549e85424745129903cd1ed262b997/users/db8febf735bd449eabafadcf14440ff7/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:05.354612 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4347f-79f8-4425-a55c-61444911ab3c admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:05.419806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4347f-79f8-4425-a55c-61444911ab3c admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:05.425027 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a144ce-608d-4073-86c5-ce0e1e6b3233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:05.427173 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a144ce-608d-4073-86c5-ce0e1e6b3233 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:05.427173 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a144ce-608d-4073-86c5-ce0e1e6b3233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:05.427173 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a144ce-608d-4073-86c5-ce0e1e6b3233 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:05.427441 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1314/2647] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:05 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:05.428626 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30428ba8-4c7b-4ca4-b7df-6036d247999f None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:05.428777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30428ba8-4c7b-4ca4-b7df-6036d247999f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:05.429144 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30428ba8-4c7b-4ca4-b7df-6036d247999f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:05.430123 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a144ce-608d-4073-86c5-ce0e1e6b3233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:05.431856 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a144ce-608d-4073-86c5-ce0e1e6b3233 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:05.444931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1334/2648] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:05.448181 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a144ce-608d-4073-86c5-ce0e1e6b3233 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:05.462489 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a144ce-608d-4073-86c5-ce0e1e6b3233 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:05.464042 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-30428ba8-4c7b-4ca4-b7df-6036d247999f None None] MFA Rules not processed for user `69c345b278e4437d98f9b1f412bbca49`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:05.465377 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-30428ba8-4c7b-4ca4-b7df-6036d247999f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:05.492608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:05.526459 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72682df0-47cd-4150-aacc-630460c6f317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:05.527453 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72682df0-47cd-4150-aacc-630460c6f317 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:05.527734 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72682df0-47cd-4150-aacc-630460c6f317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:05.528006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72682df0-47cd-4150-aacc-630460c6f317 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:05.528687 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72682df0-47cd-4150-aacc-630460c6f317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:05.531983 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72682df0-47cd-4150-aacc-630460c6f317 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:05.536663 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1315/2649] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:05.538254 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30428ba8-4c7b-4ca4-b7df-6036d247999f None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:05.550425 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30428ba8-4c7b-4ca4-b7df-6036d247999f None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:05.555305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9a08b-e016-4c74-956f-83f66f8c2309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:05.556188 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9a08b-e016-4c74-956f-83f66f8c2309 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:05.556472 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9a08b-e016-4c74-956f-83f66f8c2309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:05.556750 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9a08b-e016-4c74-956f-83f66f8c2309 admin admin] PATH_INFO: `/v3/projects/8e549e85424745129903cd1ed262b997/users/db8febf735bd449eabafadcf14440ff7/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:05.558739 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9a08b-e016-4c74-956f-83f66f8c2309 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e549e85424745129903cd1ed262b997, user_id=db8febf735bd449eabafadcf14440ff7, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:05.561408 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9a08b-e016-4c74-956f-83f66f8c2309 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:05.571801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1316/2650] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:05 2022] PUT /identity/v3/projects/8e549e85424745129903cd1ed262b997/users/db8febf735bd449eabafadcf14440ff7/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:05.572574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9a08b-e016-4c74-956f-83f66f8c2309 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:05.574117 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1dc9a08b-e016-4c74-956f-83f66f8c2309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:05.594330 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:05.632786 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9a08b-e016-4c74-956f-83f66f8c2309 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:05.636711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff6b97-6d73-4b76-9c05-36abec66912e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:05.636880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1335/2651] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:05 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:05.637349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff6b97-6d73-4b76-9c05-36abec66912e service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:05.637646 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff6b97-6d73-4b76-9c05-36abec66912e service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:05.637902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff6b97-6d73-4b76-9c05-36abec66912e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:05.641699 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a1c25-d88d-4ad6-9706-76a424d77b7f None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:05.641987 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a1c25-d88d-4ad6-9706-76a424d77b7f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:05.642265 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a1c25-d88d-4ad6-9706-76a424d77b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:05.669751 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-394a1c25-d88d-4ad6-9706-76a424d77b7f None None] MFA Rules not processed for user `db8febf735bd449eabafadcf14440ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:05.670997 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-394a1c25-d88d-4ad6-9706-76a424d77b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:05.697141 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff6b97-6d73-4b76-9c05-36abec66912e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:05.700045 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff6b97-6d73-4b76-9c05-36abec66912e service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:05.708578 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1317/2652] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:05.709422 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff6b97-6d73-4b76-9c05-36abec66912e service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:05.745389 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1336/2653] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:05.763418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a1c25-d88d-4ad6-9706-76a424d77b7f None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:05.765919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-394a1c25-d88d-4ad6-9706-76a424d77b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:05.779667 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff6b97-6d73-4b76-9c05-36abec66912e service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:05.781601 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a1c25-d88d-4ad6-9706-76a424d77b7f None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:05.784543 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534c53d-7ac4-4a5c-8121-a5ac503ca9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:05.785215 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534c53d-7ac4-4a5c-8121-a5ac503ca9ff admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:05.785355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa91103-c77c-497d-9609-26a0349200e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:05.785534 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534c53d-7ac4-4a5c-8121-a5ac503ca9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:05.785936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534c53d-7ac4-4a5c-8121-a5ac503ca9ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:05.786174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa91103-c77c-497d-9609-26a0349200e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:05.786566 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa91103-c77c-497d-9609-26a0349200e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:05.786950 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3534c53d-7ac4-4a5c-8121-a5ac503ca9ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:05.787089 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa91103-c77c-497d-9609-26a0349200e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:05.788596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3534c53d-7ac4-4a5c-8121-a5ac503ca9ff admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:05.802104 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa91103-c77c-497d-9609-26a0349200e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:05.803582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa91103-c77c-497d-9609-26a0349200e7 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:05.803858 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1318/2654] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:05.807986 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534c53d-7ac4-4a5c-8121-a5ac503ca9ff admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:05.815054 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1337/2655] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:05 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:57:05.820941 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534c53d-7ac4-4a5c-8121-a5ac503ca9ff admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:05.825816 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92701939-259b-4f4d-97d1-d0e7ab101ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:05.826513 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92701939-259b-4f4d-97d1-d0e7ab101ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:05.826823 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92701939-259b-4f4d-97d1-d0e7ab101ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:05.827976 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92701939-259b-4f4d-97d1-d0e7ab101ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:05.828713 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92701939-259b-4f4d-97d1-d0e7ab101ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:05.830306 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92701939-259b-4f4d-97d1-d0e7ab101ec4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:05.842900 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1319/2656] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:05.851341 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa91103-c77c-497d-9609-26a0349200e7 service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:05.862733 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa91103-c77c-497d-9609-26a0349200e7 service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:05.867473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913c467-c9af-4452-9c03-63796d3d3e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:05.868345 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913c467-c9af-4452-9c03-63796d3d3e71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:05.868670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913c467-c9af-4452-9c03-63796d3d3e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:05.868907 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913c467-c9af-4452-9c03-63796d3d3e71 admin admin] PATH_INFO: `/v3/projects/e4ef408d89334ec884f83ed4a97c7872/users/ff2d8316da2545edba13c38f855c290f/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:05.884591 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913c467-c9af-4452-9c03-63796d3d3e71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4ef408d89334ec884f83ed4a97c7872, user_id=ff2d8316da2545edba13c38f855c290f, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:05.887275 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913c467-c9af-4452-9c03-63796d3d3e71 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:05.978923 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1338/2657] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:05 2022] PUT /identity/v3/projects/e4ef408d89334ec884f83ed4a97c7872/users/ff2d8316da2545edba13c38f855c290f/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:05.990073 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c16bb3-8bbf-4f81-86cd-81adfeed0fb4 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:05.990619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c16bb3-8bbf-4f81-86cd-81adfeed0fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:05.990902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c16bb3-8bbf-4f81-86cd-81adfeed0fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:05.991312 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913c467-c9af-4452-9c03-63796d3d3e71 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:06.019167 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-11c16bb3-8bbf-4f81-86cd-81adfeed0fb4 None None] MFA Rules not processed for user `ff2d8316da2545edba13c38f855c290f`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:06.020388 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-11c16bb3-8bbf-4f81-86cd-81adfeed0fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:06.047224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913c467-c9af-4452-9c03-63796d3d3e71 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:06.050937 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b66327-24ec-4cb3-af92-8e8cb8bf1dc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:06.051608 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b66327-24ec-4cb3-af92-8e8cb8bf1dc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:06.051885 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b66327-24ec-4cb3-af92-8e8cb8bf1dc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:06.052159 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b66327-24ec-4cb3-af92-8e8cb8bf1dc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:06.053965 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-31b66327-24ec-4cb3-af92-8e8cb8bf1dc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:06.067119 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:06.068639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:06.090385 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1320/2658] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:06.095666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c16bb3-8bbf-4f81-86cd-81adfeed0fb4 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:06.115877 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b66327-24ec-4cb3-af92-8e8cb8bf1dc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:06.119488 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b66327-24ec-4cb3-af92-8e8cb8bf1dc9 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:06.132908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1339/2659] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:57:06.160250 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c16bb3-8bbf-4f81-86cd-81adfeed0fb4 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:06.165339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fa343-c596-4e46-8e7f-69e56e2d76c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:06.166495 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fa343-c596-4e46-8e7f-69e56e2d76c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:06.166495 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fa343-c596-4e46-8e7f-69e56e2d76c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:06.166763 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fa343-c596-4e46-8e7f-69e56e2d76c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:06.167389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fa343-c596-4e46-8e7f-69e56e2d76c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:06.169302 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fa343-c596-4e46-8e7f-69e56e2d76c7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:06.290798 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1321/2660] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:06 2022] POST /identity/v3/projects => generated 369 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:06.294729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b66327-24ec-4cb3-af92-8e8cb8bf1dc9 service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:06.306301 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b66327-24ec-4cb3-af92-8e8cb8bf1dc9 service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:06.388743 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f356f-676c-422b-b63d-e47a637de5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:06.390136 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f356f-676c-422b-b63d-e47a637de5c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:06.390888 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f356f-676c-422b-b63d-e47a637de5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:06.391547 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f356f-676c-422b-b63d-e47a637de5c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:06.392502 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f356f-676c-422b-b63d-e47a637de5c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:06.394733 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f356f-676c-422b-b63d-e47a637de5c4 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:06.503238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1340/2661] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:06 2022] POST /identity/v3/users => generated 291 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:06.504292 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fa343-c596-4e46-8e7f-69e56e2d76c7 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:06.523369 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fa343-c596-4e46-8e7f-69e56e2d76c7 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:06.526049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e0b80-9ca4-4b50-8a58-2f74fedcb4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:06.526771 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e0b80-9ca4-4b50-8a58-2f74fedcb4f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:06.527698 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e0b80-9ca4-4b50-8a58-2f74fedcb4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:06.527978 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e0b80-9ca4-4b50-8a58-2f74fedcb4f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:06.528721 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e0b80-9ca4-4b50-8a58-2f74fedcb4f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:06.530941 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e0b80-9ca4-4b50-8a58-2f74fedcb4f9 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:06.542470 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1322/2662] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:06.547308 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f356f-676c-422b-b63d-e47a637de5c4 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:06.558203 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f356f-676c-422b-b63d-e47a637de5c4 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:06.563466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18718d84-785e-4198-b39d-09692d1850cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:06.564368 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18718d84-785e-4198-b39d-09692d1850cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:06.564648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18718d84-785e-4198-b39d-09692d1850cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:06.565054 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18718d84-785e-4198-b39d-09692d1850cf admin admin] PATH_INFO: `/v3/projects/9f468b06dd9d43f183fa578a7e5d239a/users/d9dcfae0c47542ae9968d09ef6357dee/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:06.581156 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18718d84-785e-4198-b39d-09692d1850cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f468b06dd9d43f183fa578a7e5d239a, user_id=d9dcfae0c47542ae9968d09ef6357dee, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:06.583819 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18718d84-785e-4198-b39d-09692d1850cf admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:06.656598 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e0b80-9ca4-4b50-8a58-2f74fedcb4f9 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:06.663394 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1341/2663] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:06 2022] PUT /identity/v3/projects/9f468b06dd9d43f183fa578a7e5d239a/users/d9dcfae0c47542ae9968d09ef6357dee/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:06.664666 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18718d84-785e-4198-b39d-09692d1850cf admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:06.666711 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e0b80-9ca4-4b50-8a58-2f74fedcb4f9 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:06.725142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5bd21-d6f4-4e9b-b816-c34d0280171a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:06.726114 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5bd21-d6f4-4e9b-b816-c34d0280171a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:06.726399 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5bd21-d6f4-4e9b-b816-c34d0280171a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:06.726734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5bd21-d6f4-4e9b-b816-c34d0280171a admin admin] PATH_INFO: `/v3/users/e5db0e1d7dfb4f8d9aff3614a5d6a06a` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:06.728002 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5bd21-d6f4-4e9b-b816-c34d0280171a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5db0e1d7dfb4f8d9aff3614a5d6a06a)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:06.728162 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18718d84-785e-4198-b39d-09692d1850cf admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:06.729926 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5bd21-d6f4-4e9b-b816-c34d0280171a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:06.733097 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2a504-0154-41e5-9223-b02ae01af193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:06.734304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2a504-0154-41e5-9223-b02ae01af193 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:06.734710 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2a504-0154-41e5-9223-b02ae01af193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:06.734986 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2a504-0154-41e5-9223-b02ae01af193 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:06.735713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf2a504-0154-41e5-9223-b02ae01af193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:06.737301 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf2a504-0154-41e5-9223-b02ae01af193 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:06.747360 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1342/2664] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:06.751594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2a504-0154-41e5-9223-b02ae01af193 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:06.761602 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2a504-0154-41e5-9223-b02ae01af193 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:06.766894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5577382-9db9-4159-a484-9dd4b8a487d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:06.767759 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5577382-9db9-4159-a484-9dd4b8a487d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:06.768069 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5577382-9db9-4159-a484-9dd4b8a487d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:06.768344 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5577382-9db9-4159-a484-9dd4b8a487d7 admin admin] PATH_INFO: `/v3/projects/9f468b06dd9d43f183fa578a7e5d239a/users/d9dcfae0c47542ae9968d09ef6357dee/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:06.770282 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5577382-9db9-4159-a484-9dd4b8a487d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f468b06dd9d43f183fa578a7e5d239a, user_id=d9dcfae0c47542ae9968d09ef6357dee, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:06.773095 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5577382-9db9-4159-a484-9dd4b8a487d7 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:06.783736 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1343/2665] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:06 2022] PUT /identity/v3/projects/9f468b06dd9d43f183fa578a7e5d239a/users/d9dcfae0c47542ae9968d09ef6357dee/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:06.795043 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77754449-eaa6-4f4f-907c-d628b5492761 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:06.795272 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77754449-eaa6-4f4f-907c-d628b5492761 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:06.795552 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77754449-eaa6-4f4f-907c-d628b5492761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:06.802291 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:06.821258 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-77754449-eaa6-4f4f-907c-d628b5492761 None None] MFA Rules not processed for user `d9dcfae0c47542ae9968d09ef6357dee`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:06.822452 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-77754449-eaa6-4f4f-907c-d628b5492761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:06.890240 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1344/2666] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:06 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:06.901648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77754449-eaa6-4f4f-907c-d628b5492761 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:06.937395 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-42d5bd21-d6f4-4e9b-b816-c34d0280171a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5db0e1d7dfb4f8d9aff3614a5d6a06a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:06.945069 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-42d5bd21-d6f4-4e9b-b816-c34d0280171a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5db0e1d7dfb4f8d9aff3614a5d6a06a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:06.945774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-42d5bd21-d6f4-4e9b-b816-c34d0280171a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5db0e1d7dfb4f8d9aff3614a5d6a06a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:06.946436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-42d5bd21-d6f4-4e9b-b816-c34d0280171a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5db0e1d7dfb4f8d9aff3614a5d6a06a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:06.951582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:06.958559 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77754449-eaa6-4f4f-907c-d628b5492761 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:06.959945 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-42d5bd21-d6f4-4e9b-b816-c34d0280171a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5db0e1d7dfb4f8d9aff3614a5d6a06a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:06.962707 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7bdd1-945b-43a1-843a-3532075f9bf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:06.963363 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7bdd1-945b-43a1-843a-3532075f9bf5 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:06.963647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7bdd1-945b-43a1-843a-3532075f9bf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:06.963986 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7bdd1-945b-43a1-843a-3532075f9bf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:06.965841 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f5f7bdd1-945b-43a1-843a-3532075f9bf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:06.978446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7bdd1-945b-43a1-843a-3532075f9bf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:06.979912 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7bdd1-945b-43a1-843a-3532075f9bf5 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:06.989624 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1345/2667] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:06 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:57:07.094147 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1323/2668] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:06 2022] DELETE /identity/v3/users/e5db0e1d7dfb4f8d9aff3614a5d6a06a => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:07.098828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af06a7-d780-4d5d-8bd8-f3ce0e1c82c2 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:07.099122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af06a7-d780-4d5d-8bd8-f3ce0e1c82c2 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:07.099401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af06a7-d780-4d5d-8bd8-f3ce0e1c82c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:07.117646 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-f1af06a7-d780-4d5d-8bd8-f3ce0e1c82c2 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:07.118901 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f1af06a7-d780-4d5d-8bd8-f3ce0e1c82c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:07.183521 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1346/2669] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:07.194120 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5bd21-d6f4-4e9b-b816-c34d0280171a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:07.195705 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-42d5bd21-d6f4-4e9b-b816-c34d0280171a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:07.257995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5bd21-d6f4-4e9b-b816-c34d0280171a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:07.261701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5a6ba-be2d-44ac-88b0-d47255bbf68c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:07.262369 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5a6ba-be2d-44ac-88b0-d47255bbf68c service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:07.262683 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5a6ba-be2d-44ac-88b0-d47255bbf68c service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:07.262957 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5a6ba-be2d-44ac-88b0-d47255bbf68c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:07.273071 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5a6ba-be2d-44ac-88b0-d47255bbf68c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:07.274495 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5a6ba-be2d-44ac-88b0-d47255bbf68c service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:07.284508 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1324/2670] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:07.497397 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94ce47-303e-4666-879b-4b631383f1bb None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:07.497869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94ce47-303e-4666-879b-4b631383f1bb None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:07.497869 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94ce47-303e-4666-879b-4b631383f1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:07.518857 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-ff94ce47-303e-4666-879b-4b631383f1bb None None] MFA Rules not processed for user `69c345b278e4437d98f9b1f412bbca49`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:07.519071 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-ff94ce47-303e-4666-879b-4b631383f1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:07.550524 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=100 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:07.577474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1347/2671] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:07.742568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5a6ba-be2d-44ac-88b0-d47255bbf68c service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:07.744241 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-75f5a6ba-be2d-44ac-88b0-d47255bbf68c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:07.757853 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5a6ba-be2d-44ac-88b0-d47255bbf68c service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:07.762615 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087e667-1955-4813-a7e0-ddb256a1e9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:07.763484 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087e667-1955-4813-a7e0-ddb256a1e9f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:07.763872 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087e667-1955-4813-a7e0-ddb256a1e9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:07.764049 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087e667-1955-4813-a7e0-ddb256a1e9f9 admin admin] PATH_INFO: `/v3/projects/cd4d22efb11c4436a4fdbcf0b01a3834` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:07.765224 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3087e667-1955-4813-a7e0-ddb256a1e9f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd4d22efb11c4436a4fdbcf0b01a3834)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:07.767354 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3087e667-1955-4813-a7e0-ddb256a1e9f9 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:07.837791 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94ce47-303e-4666-879b-4b631383f1bb None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:07.849188 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94ce47-303e-4666-879b-4b631383f1bb None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:07.854347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddf033-ea28-4993-9995-b3bddb55a07d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:07.854347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddf033-ea28-4993-9995-b3bddb55a07d service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:07.854347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddf033-ea28-4993-9995-b3bddb55a07d service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:07.854347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddf033-ea28-4993-9995-b3bddb55a07d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:07.855989 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b8ddf033-ea28-4993-9995-b3bddb55a07d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:07.866879 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ddf033-ea28-4993-9995-b3bddb55a07d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:07.868316 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ddf033-ea28-4993-9995-b3bddb55a07d service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:07.876735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1348/2672] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:57:07.889696 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3087e667-1955-4813-a7e0-ddb256a1e9f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd4d22efb11c4436a4fdbcf0b01a3834', 'request_id': 'req-3087e667-1955-4813-a7e0-ddb256a1e9f9'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:07.897103 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3087e667-1955-4813-a7e0-ddb256a1e9f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd4d22efb11c4436a4fdbcf0b01a3834', 'request_id': 'req-3087e667-1955-4813-a7e0-ddb256a1e9f9'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:07.984000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3087e667-1955-4813-a7e0-ddb256a1e9f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd4d22efb11c4436a4fdbcf0b01a3834', 'request_id': 'req-3087e667-1955-4813-a7e0-ddb256a1e9f9'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:07.997701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3087e667-1955-4813-a7e0-ddb256a1e9f9 admin admin] The token cache is being invalidate because project cd4d22efb11c4436a4fdbcf0b01a3834 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:07.997986 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3087e667-1955-4813-a7e0-ddb256a1e9f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:07.998675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3087e667-1955-4813-a7e0-ddb256a1e9f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:08.000311 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1325/2673] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:07 2022] DELETE /identity/v3/projects/cd4d22efb11c4436a4fdbcf0b01a3834 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:08.004285 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddf033-ea28-4993-9995-b3bddb55a07d service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:08.015180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddf033-ea28-4993-9995-b3bddb55a07d service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:08.056812 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087e667-1955-4813-a7e0-ddb256a1e9f9 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:08.058566 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3087e667-1955-4813-a7e0-ddb256a1e9f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:08.073349 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8794bf8-6bc9-43c9-9e7f-0fd5a26a688c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:08.074235 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8794bf8-6bc9-43c9-9e7f-0fd5a26a688c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:08.074515 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8794bf8-6bc9-43c9-9e7f-0fd5a26a688c admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:08.075068 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8794bf8-6bc9-43c9-9e7f-0fd5a26a688c admin admin] PATH_INFO: `/v3/users/db8febf735bd449eabafadcf14440ff7` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:08.076301 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8794bf8-6bc9-43c9-9e7f-0fd5a26a688c admin admin] RBAC: Authorizing `identity:delete_user(user_id=db8febf735bd449eabafadcf14440ff7)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:08.078248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8794bf8-6bc9-43c9-9e7f-0fd5a26a688c admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:08.137284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087e667-1955-4813-a7e0-ddb256a1e9f9 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:08.142789 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecb766-8adf-43fd-a634-54d1a1b7658e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:08.143668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecb766-8adf-43fd-a634-54d1a1b7658e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:08.148389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecb766-8adf-43fd-a634-54d1a1b7658e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:08.148389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecb766-8adf-43fd-a634-54d1a1b7658e admin admin] PATH_INFO: `/v3/users/ff2d8316da2545edba13c38f855c290f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:08.148389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ecb766-8adf-43fd-a634-54d1a1b7658e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff2d8316da2545edba13c38f855c290f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:08.148389 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ecb766-8adf-43fd-a634-54d1a1b7658e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:08.152292 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:08.397478 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-54ecb766-8adf-43fd-a634-54d1a1b7658e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff2d8316da2545edba13c38f855c290f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:08.410052 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-54ecb766-8adf-43fd-a634-54d1a1b7658e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff2d8316da2545edba13c38f855c290f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:08.410871 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-54ecb766-8adf-43fd-a634-54d1a1b7658e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff2d8316da2545edba13c38f855c290f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:08.411483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-54ecb766-8adf-43fd-a634-54d1a1b7658e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff2d8316da2545edba13c38f855c290f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:08.415628 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d8794bf8-6bc9-43c9-9e7f-0fd5a26a688c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db8febf735bd449eabafadcf14440ff7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:08.423858 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d8794bf8-6bc9-43c9-9e7f-0fd5a26a688c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db8febf735bd449eabafadcf14440ff7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:08.426347 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-54ecb766-8adf-43fd-a634-54d1a1b7658e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff2d8316da2545edba13c38f855c290f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:08.427007 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d8794bf8-6bc9-43c9-9e7f-0fd5a26a688c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db8febf735bd449eabafadcf14440ff7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:08.427693 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d8794bf8-6bc9-43c9-9e7f-0fd5a26a688c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db8febf735bd449eabafadcf14440ff7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:08.445278 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d8794bf8-6bc9-43c9-9e7f-0fd5a26a688c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db8febf735bd449eabafadcf14440ff7'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:08.499410 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1326/2674] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:08 2022] DELETE /identity/v3/users/ff2d8316da2545edba13c38f855c290f => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:08.503920 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86107333-931c-4f12-b611-4b6507dd9544 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:08.504315 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86107333-931c-4f12-b611-4b6507dd9544 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:08.504598 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86107333-931c-4f12-b611-4b6507dd9544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:08.522964 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-86107333-931c-4f12-b611-4b6507dd9544 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:08.525338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-86107333-931c-4f12-b611-4b6507dd9544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:08.574459 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1349/2675] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:08 2022] DELETE /identity/v3/users/db8febf735bd449eabafadcf14440ff7 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:08.584225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1327/2676] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:08.593857 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8794bf8-6bc9-43c9-9e7f-0fd5a26a688c admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:08.596041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d8794bf8-6bc9-43c9-9e7f-0fd5a26a688c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:08.646008 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:08.658245 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8794bf8-6bc9-43c9-9e7f-0fd5a26a688c admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:08.661844 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19cdb0e-c43e-4d65-8592-ca7a07216384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:08.681630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cdb0e-c43e-4d65-8592-ca7a07216384 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:08.681630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cdb0e-c43e-4d65-8592-ca7a07216384 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:08.681630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cdb0e-c43e-4d65-8592-ca7a07216384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:08.723646 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19cdb0e-c43e-4d65-8592-ca7a07216384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:08.725115 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19cdb0e-c43e-4d65-8592-ca7a07216384 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:08.734700 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1350/2677] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:08.993661 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86107333-931c-4f12-b611-4b6507dd9544 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:08.996060 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-86107333-931c-4f12-b611-4b6507dd9544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:09.010366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86107333-931c-4f12-b611-4b6507dd9544 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:09.016121 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6653f-9f25-4ddc-858c-a2ff765e92ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:09.030403 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6653f-9f25-4ddc-858c-a2ff765e92ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:09.030563 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6653f-9f25-4ddc-858c-a2ff765e92ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:09.030846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6653f-9f25-4ddc-858c-a2ff765e92ab admin admin] PATH_INFO: `/v3/projects/8e549e85424745129903cd1ed262b997` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:09.033933 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6653f-9f25-4ddc-858c-a2ff765e92ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e549e85424745129903cd1ed262b997)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:09.036066 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6653f-9f25-4ddc-858c-a2ff765e92ab admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:09.156684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-26c6653f-9f25-4ddc-858c-a2ff765e92ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e549e85424745129903cd1ed262b997', 'request_id': 'req-26c6653f-9f25-4ddc-858c-a2ff765e92ab'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:09.162861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-26c6653f-9f25-4ddc-858c-a2ff765e92ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e549e85424745129903cd1ed262b997', 'request_id': 'req-26c6653f-9f25-4ddc-858c-a2ff765e92ab'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:09.181761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19cdb0e-c43e-4d65-8592-ca7a07216384 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:09.183181 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a19cdb0e-c43e-4d65-8592-ca7a07216384 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:09.196994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19cdb0e-c43e-4d65-8592-ca7a07216384 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:09.259401 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f12471-89f9-4d50-9c0c-966f994a6081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:09.260264 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f12471-89f9-4d50-9c0c-966f994a6081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:09.260549 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f12471-89f9-4d50-9c0c-966f994a6081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:09.260827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f12471-89f9-4d50-9c0c-966f994a6081 admin admin] PATH_INFO: `/v3/projects/e4ef408d89334ec884f83ed4a97c7872` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:09.261996 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f12471-89f9-4d50-9c0c-966f994a6081 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4ef408d89334ec884f83ed4a97c7872)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:09.264034 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f12471-89f9-4d50-9c0c-966f994a6081 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:09.333211 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-26c6653f-9f25-4ddc-858c-a2ff765e92ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e549e85424745129903cd1ed262b997', 'request_id': 'req-26c6653f-9f25-4ddc-858c-a2ff765e92ab'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:09.345556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-26c6653f-9f25-4ddc-858c-a2ff765e92ab admin admin] The token cache is being invalidate because project 8e549e85424745129903cd1ed262b997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:09.345847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-26c6653f-9f25-4ddc-858c-a2ff765e92ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:09.346504 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-26c6653f-9f25-4ddc-858c-a2ff765e92ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:09.348069 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1328/2678] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:08 2022] DELETE /identity/v3/projects/8e549e85424745129903cd1ed262b997 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:09.352600 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80242be-c811-4619-88dc-9536d05a16f4 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:09.352892 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80242be-c811-4619-88dc-9536d05a16f4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:09.353173 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80242be-c811-4619-88dc-9536d05a16f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:09.367757 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:09.373092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-e80242be-c811-4619-88dc-9536d05a16f4 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:09.374298 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e80242be-c811-4619-88dc-9536d05a16f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:09.382468 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1329/2679] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:09.389689 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3ac40-1148-4bd7-8d2f-29091aa2a21e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:09.393689 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3ac40-1148-4bd7-8d2f-29091aa2a21e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:09.393689 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3ac40-1148-4bd7-8d2f-29091aa2a21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:09.408543 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-90f3ac40-1148-4bd7-8d2f-29091aa2a21e None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:09.409698 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-90f3ac40-1148-4bd7-8d2f-29091aa2a21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:09.420871 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1330/2680] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:09.425588 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3ac40-1148-4bd7-8d2f-29091aa2a21e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:09.432991 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-90f12471-89f9-4d50-9c0c-966f994a6081 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4ef408d89334ec884f83ed4a97c7872', 'request_id': 'req-90f12471-89f9-4d50-9c0c-966f994a6081'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:09.434478 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3ac40-1148-4bd7-8d2f-29091aa2a21e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:09.439633 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-90f12471-89f9-4d50-9c0c-966f994a6081 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4ef408d89334ec884f83ed4a97c7872', 'request_id': 'req-90f12471-89f9-4d50-9c0c-966f994a6081'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:09.491686 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5dce31-0c33-4f6b-9f25-1ee477dee746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:09.492361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5dce31-0c33-4f6b-9f25-1ee477dee746 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:09.492653 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5dce31-0c33-4f6b-9f25-1ee477dee746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:09.492920 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5dce31-0c33-4f6b-9f25-1ee477dee746 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:09.493647 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5dce31-0c33-4f6b-9f25-1ee477dee746 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:57:09.493956 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5dce31-0c33-4f6b-9f25-1ee477dee746 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:09.495876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5dce31-0c33-4f6b-9f25-1ee477dee746 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:09.508415 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1331/2681] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:57:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:57:09.512773 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5dce31-0c33-4f6b-9f25-1ee477dee746 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:09.522182 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-90f12471-89f9-4d50-9c0c-966f994a6081 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4ef408d89334ec884f83ed4a97c7872', 'request_id': 'req-90f12471-89f9-4d50-9c0c-966f994a6081'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:09.526862 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5dce31-0c33-4f6b-9f25-1ee477dee746 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:09.536411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-90f12471-89f9-4d50-9c0c-966f994a6081 admin admin] The token cache is being invalidate because project e4ef408d89334ec884f83ed4a97c7872 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:09.536811 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-90f12471-89f9-4d50-9c0c-966f994a6081 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:09.537551 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-90f12471-89f9-4d50-9c0c-966f994a6081 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:09.538324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94231a0d-c7e5-431d-9cbb-87ddfa65b6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:09.539063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94231a0d-c7e5-431d-9cbb-87ddfa65b6c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:09.539347 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94231a0d-c7e5-431d-9cbb-87ddfa65b6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:09.539628 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94231a0d-c7e5-431d-9cbb-87ddfa65b6c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:09.539772 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1351/2682] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:09 2022] DELETE /identity/v3/projects/e4ef408d89334ec884f83ed4a97c7872 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:09.540334 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94231a0d-c7e5-431d-9cbb-87ddfa65b6c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:09.542237 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94231a0d-c7e5-431d-9cbb-87ddfa65b6c3 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:09.543404 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f12471-89f9-4d50-9c0c-966f994a6081 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:09.544683 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-90f12471-89f9-4d50-9c0c-966f994a6081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:09.557941 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f12471-89f9-4d50-9c0c-966f994a6081 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:09.562465 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9002301-7b64-45ec-8fe9-f966c59de954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:09.563912 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9002301-7b64-45ec-8fe9-f966c59de954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:09.564174 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9002301-7b64-45ec-8fe9-f966c59de954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:09.564443 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9002301-7b64-45ec-8fe9-f966c59de954 admin admin] PATH_INFO: `/v3/users/d9dcfae0c47542ae9968d09ef6357dee` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:09.565632 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9002301-7b64-45ec-8fe9-f966c59de954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9dcfae0c47542ae9968d09ef6357dee)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:09.567569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9002301-7b64-45ec-8fe9-f966c59de954 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:09.588347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:09.694467 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1332/2683] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:09 2022] POST /identity/v3/projects => generated 375 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:09.698638 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94231a0d-c7e5-431d-9cbb-87ddfa65b6c3 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:09.700199 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-94231a0d-c7e5-431d-9cbb-87ddfa65b6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:09.763915 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94231a0d-c7e5-431d-9cbb-87ddfa65b6c3 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:09.769419 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71faa173-c99c-4268-bed7-d8933af72750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:09.770100 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faa173-c99c-4268-bed7-d8933af72750 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:09.770366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faa173-c99c-4268-bed7-d8933af72750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:09.770675 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faa173-c99c-4268-bed7-d8933af72750 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:09.771361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71faa173-c99c-4268-bed7-d8933af72750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:09.773269 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71faa173-c99c-4268-bed7-d8933af72750 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:09.879650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f9002301-7b64-45ec-8fe9-f966c59de954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9dcfae0c47542ae9968d09ef6357dee'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:09.886840 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f9002301-7b64-45ec-8fe9-f966c59de954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9dcfae0c47542ae9968d09ef6357dee'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:09.887589 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f9002301-7b64-45ec-8fe9-f966c59de954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9dcfae0c47542ae9968d09ef6357dee'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:09.888435 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f9002301-7b64-45ec-8fe9-f966c59de954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9dcfae0c47542ae9968d09ef6357dee'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:09.900720 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-f9002301-7b64-45ec-8fe9-f966c59de954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9dcfae0c47542ae9968d09ef6357dee'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:09.990724 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1333/2684] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:09 2022] POST /identity/v3/users => generated 288 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:09.994819 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71faa173-c99c-4268-bed7-d8933af72750 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:09.996495 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-71faa173-c99c-4268-bed7-d8933af72750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:10.009115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71faa173-c99c-4268-bed7-d8933af72750 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:10.013691 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ebf22-4193-450b-b90d-3700f6cae818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:10.014326 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ebf22-4193-450b-b90d-3700f6cae818 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.014634 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ebf22-4193-450b-b90d-3700f6cae818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.014910 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ebf22-4193-450b-b90d-3700f6cae818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.015618 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ebf22-4193-450b-b90d-3700f6cae818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:10.020560 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ebf22-4193-450b-b90d-3700f6cae818 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:10.024953 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1352/2685] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:09 2022] DELETE /identity/v3/users/d9dcfae0c47542ae9968d09ef6357dee => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:10.032657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1334/2686] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:10.035939 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9002301-7b64-45ec-8fe9-f966c59de954 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:10.097905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9002301-7b64-45ec-8fe9-f966c59de954 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:10.102564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf4bae-5f2e-4541-8274-a345b48c041e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:10.103230 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf4bae-5f2e-4541-8274-a345b48c041e admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.103512 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf4bae-5f2e-4541-8274-a345b48c041e admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.103790 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf4bae-5f2e-4541-8274-a345b48c041e admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.104661 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf4bae-5f2e-4541-8274-a345b48c041e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:10.106550 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf4bae-5f2e-4541-8274-a345b48c041e admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:10.117567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1353/2687] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:10.121877 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ebf22-4193-450b-b90d-3700f6cae818 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:10.132764 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ebf22-4193-450b-b90d-3700f6cae818 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:10.137871 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a47af4-5334-411e-ad12-2dcf4ab8e5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:10.138793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a47af4-5334-411e-ad12-2dcf4ab8e5d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.139073 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a47af4-5334-411e-ad12-2dcf4ab8e5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.139346 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a47af4-5334-411e-ad12-2dcf4ab8e5d9 admin admin] PATH_INFO: `/v3/projects/72359b7d3cdf4a9a8bec00cbf1e1aa17/users/53e27bd783c74442a72aa93ce00bedf7/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.154872 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a47af4-5334-411e-ad12-2dcf4ab8e5d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72359b7d3cdf4a9a8bec00cbf1e1aa17, user_id=53e27bd783c74442a72aa93ce00bedf7, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:10.158217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a47af4-5334-411e-ad12-2dcf4ab8e5d9 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:10.283501 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1335/2688] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:10 2022] PUT /identity/v3/projects/72359b7d3cdf4a9a8bec00cbf1e1aa17/users/53e27bd783c74442a72aa93ce00bedf7/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:10.299052 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1384818-e146-4ccd-84da-0faee6ac3d5f None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.299429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1384818-e146-4ccd-84da-0faee6ac3d5f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.299718 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1384818-e146-4ccd-84da-0faee6ac3d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.328084 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-f1384818-e146-4ccd-84da-0faee6ac3d5f None None] MFA Rules not processed for user `53e27bd783c74442a72aa93ce00bedf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:10.329385 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f1384818-e146-4ccd-84da-0faee6ac3d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:10.400817 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1354/2689] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:10.405372 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a47af4-5334-411e-ad12-2dcf4ab8e5d9 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:10.455824 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1384818-e146-4ccd-84da-0faee6ac3d5f None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:10.457713 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f1384818-e146-4ccd-84da-0faee6ac3d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:10.470084 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a47af4-5334-411e-ad12-2dcf4ab8e5d9 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:10.479648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1384818-e146-4ccd-84da-0faee6ac3d5f None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:10.481723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01d619-857b-45f6-94c0-49bd91434266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:10.482223 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01d619-857b-45f6-94c0-49bd91434266 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.482625 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01d619-857b-45f6-94c0-49bd91434266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.482907 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01d619-857b-45f6-94c0-49bd91434266 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.483784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f01d619-857b-45f6-94c0-49bd91434266 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:10.487360 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f01d619-857b-45f6-94c0-49bd91434266 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:10.493097 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9aac7f-162a-4094-b8ee-368d801b382b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:10.494026 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9aac7f-162a-4094-b8ee-368d801b382b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.494310 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9aac7f-162a-4094-b8ee-368d801b382b admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.494611 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9aac7f-162a-4094-b8ee-368d801b382b admin admin] PATH_INFO: `/v3/projects/9f468b06dd9d43f183fa578a7e5d239a` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.495963 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9aac7f-162a-4094-b8ee-368d801b382b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f468b06dd9d43f183fa578a7e5d239a)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:10.497935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9aac7f-162a-4094-b8ee-368d801b382b admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:10.649929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1336/2690] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:10 2022] POST /identity/v3/projects => generated 375 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:10.654574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01d619-857b-45f6-94c0-49bd91434266 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:10.700846 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-9f9aac7f-162a-4094-b8ee-368d801b382b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f468b06dd9d43f183fa578a7e5d239a', 'request_id': 'req-9f9aac7f-162a-4094-b8ee-368d801b382b'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:10.707672 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-9f9aac7f-162a-4094-b8ee-368d801b382b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f468b06dd9d43f183fa578a7e5d239a', 'request_id': 'req-9f9aac7f-162a-4094-b8ee-368d801b382b'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:10.717224 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01d619-857b-45f6-94c0-49bd91434266 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:10.773158 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f38f1-fd60-45b6-982a-b93275b98084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:10.773831 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f38f1-fd60-45b6-982a-b93275b98084 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.774143 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f38f1-fd60-45b6-982a-b93275b98084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.774398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f38f1-fd60-45b6-982a-b93275b98084 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.775151 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f38f1-fd60-45b6-982a-b93275b98084 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:10.786462 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f38f1-fd60-45b6-982a-b93275b98084 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:10.835790 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-9f9aac7f-162a-4094-b8ee-368d801b382b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f468b06dd9d43f183fa578a7e5d239a', 'request_id': 'req-9f9aac7f-162a-4094-b8ee-368d801b382b'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:10.847361 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-9f9aac7f-162a-4094-b8ee-368d801b382b admin admin] The token cache is being invalidate because project 9f468b06dd9d43f183fa578a7e5d239a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:10.847647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-9f9aac7f-162a-4094-b8ee-368d801b382b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:10.848282 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-9f9aac7f-162a-4094-b8ee-368d801b382b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:10.849829 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1355/2691] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:10 2022] DELETE /identity/v3/projects/9f468b06dd9d43f183fa578a7e5d239a => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:10.855343 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e8aa2-a526-49ab-9f7c-296bcc7b7ce5 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.855630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e8aa2-a526-49ab-9f7c-296bcc7b7ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.855909 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e8aa2-a526-49ab-9f7c-296bcc7b7ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.874501 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c19e8aa2-a526-49ab-9f7c-296bcc7b7ce5 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:10.876463 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c19e8aa2-a526-49ab-9f7c-296bcc7b7ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:10.884624 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1356/2692] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:10.893474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f61307-9574-405d-a553-9ba054c43355 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.893766 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f61307-9574-405d-a553-9ba054c43355 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.894048 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f61307-9574-405d-a553-9ba054c43355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.912408 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c1f61307-9574-405d-a553-9ba054c43355 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:10.913592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c1f61307-9574-405d-a553-9ba054c43355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:10.921574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1357/2693] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:10.925833 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f61307-9574-405d-a553-9ba054c43355 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:10.934271 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1337/2694] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:10 2022] POST /identity/v3/users => generated 294 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:10.934617 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f61307-9574-405d-a553-9ba054c43355 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:10.938669 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f38f1-fd60-45b6-982a-b93275b98084 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:10.940107 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca053b1d-5d85-409c-8969-8e659be3557f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:10.940394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-222f38f1-fd60-45b6-982a-b93275b98084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:10.940753 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca053b1d-5d85-409c-8969-8e659be3557f admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.941037 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca053b1d-5d85-409c-8969-8e659be3557f admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.941344 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca053b1d-5d85-409c-8969-8e659be3557f admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.942066 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca053b1d-5d85-409c-8969-8e659be3557f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:57:10.942376 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca053b1d-5d85-409c-8969-8e659be3557f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:10.943995 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca053b1d-5d85-409c-8969-8e659be3557f admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:10.961341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f38f1-fd60-45b6-982a-b93275b98084 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:10.961491 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1358/2695] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:57:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:57:10.961590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c193a47-2bd6-4a31-b73f-a970f0b8bf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:10.961590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c193a47-2bd6-4a31-b73f-a970f0b8bf60 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.961590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c193a47-2bd6-4a31-b73f-a970f0b8bf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.961590 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c193a47-2bd6-4a31-b73f-a970f0b8bf60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.961956 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c193a47-2bd6-4a31-b73f-a970f0b8bf60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:10.963505 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c193a47-2bd6-4a31-b73f-a970f0b8bf60 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:10.963969 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca053b1d-5d85-409c-8969-8e659be3557f admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:10.973665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca053b1d-5d85-409c-8969-8e659be3557f admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:10.977619 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1338/2696] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:10.978699 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c193a47-2bd6-4a31-b73f-a970f0b8bf60 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:10.979292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6c2eb-c605-4f35-9aac-985194a7a6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:10.979952 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6c2eb-c605-4f35-9aac-985194a7a6e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.980234 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6c2eb-c605-4f35-9aac-985194a7a6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.980510 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6c2eb-c605-4f35-9aac-985194a7a6e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.981197 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6c2eb-c605-4f35-9aac-985194a7a6e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:10.983383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6c2eb-c605-4f35-9aac-985194a7a6e5 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:10.989142 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c193a47-2bd6-4a31-b73f-a970f0b8bf60 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:10.993929 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56283463-b439-4068-9b71-013762d6cb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:10.997430 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56283463-b439-4068-9b71-013762d6cb4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.997430 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56283463-b439-4068-9b71-013762d6cb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.997430 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56283463-b439-4068-9b71-013762d6cb4a admin admin] PATH_INFO: `/v3/projects/5b6dc31874b2442f9029bf18b21b30a9/users/ebeb8f549a8648c4ac513aa66e761a1c/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.011355 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56283463-b439-4068-9b71-013762d6cb4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b6dc31874b2442f9029bf18b21b30a9, user_id=ebeb8f549a8648c4ac513aa66e761a1c, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:11.014032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56283463-b439-4068-9b71-013762d6cb4a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:11.077728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1359/2697] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:10 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:11.082319 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6c2eb-c605-4f35-9aac-985194a7a6e5 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:11.106633 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:11.114826 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1339/2698] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:10 2022] PUT /identity/v3/projects/5b6dc31874b2442f9029bf18b21b30a9/users/ebeb8f549a8648c4ac513aa66e761a1c/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:11.118631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56283463-b439-4068-9b71-013762d6cb4a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:11.146941 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6c2eb-c605-4f35-9aac-985194a7a6e5 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:11.189305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56283463-b439-4068-9b71-013762d6cb4a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:11.194936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d117d-f916-4c04-9db0-e3a4c66c1a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:11.195745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d117d-f916-4c04-9db0-e3a4c66c1a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:11.196037 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d117d-f916-4c04-9db0-e3a4c66c1a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:11.196317 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d117d-f916-4c04-9db0-e3a4c66c1a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.197080 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d117d-f916-4c04-9db0-e3a4c66c1a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:11.198833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d117d-f916-4c04-9db0-e3a4c66c1a70 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:11.210687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1340/2699] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:11.214449 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8d580-77f3-4064-801c-308143dd659e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:11.214856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d117d-f916-4c04-9db0-e3a4c66c1a70 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:11.215180 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8d580-77f3-4064-801c-308143dd659e admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:11.215492 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8d580-77f3-4064-801c-308143dd659e admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:11.215767 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8d580-77f3-4064-801c-308143dd659e admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.216458 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8d580-77f3-4064-801c-308143dd659e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:11.218448 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8d580-77f3-4064-801c-308143dd659e admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:11.229285 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d117d-f916-4c04-9db0-e3a4c66c1a70 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:11.235797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653aa194-64df-4011-92b6-156fd9e2e906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:11.236863 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653aa194-64df-4011-92b6-156fd9e2e906 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:11.237178 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653aa194-64df-4011-92b6-156fd9e2e906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:11.237457 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653aa194-64df-4011-92b6-156fd9e2e906 admin admin] PATH_INFO: `/v3/projects/5b6dc31874b2442f9029bf18b21b30a9/users/ebeb8f549a8648c4ac513aa66e761a1c/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.239611 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653aa194-64df-4011-92b6-156fd9e2e906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b6dc31874b2442f9029bf18b21b30a9, user_id=ebeb8f549a8648c4ac513aa66e761a1c, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:11.242526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653aa194-64df-4011-92b6-156fd9e2e906 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:11.254670 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1341/2700] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:11 2022] PUT /identity/v3/projects/5b6dc31874b2442f9029bf18b21b30a9/users/ebeb8f549a8648c4ac513aa66e761a1c/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:11.266559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0cd75-8408-48c1-b2ea-c2603bc203bf None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:11.266951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0cd75-8408-48c1-b2ea-c2603bc203bf None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:11.267309 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0cd75-8408-48c1-b2ea-c2603bc203bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.300744 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-4df0cd75-8408-48c1-b2ea-c2603bc203bf None None] MFA Rules not processed for user `ebeb8f549a8648c4ac513aa66e761a1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:11.301951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4df0cd75-8408-48c1-b2ea-c2603bc203bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:11.352941 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1360/2701] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:11 2022] POST /identity/v3/users => generated 294 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:11.354866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8d580-77f3-4064-801c-308143dd659e admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:11.377001 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1342/2702] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:11.378708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8d580-77f3-4064-801c-308143dd659e admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:11.380903 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:11.389025 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0cd75-8408-48c1-b2ea-c2603bc203bf None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:11.392729 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4df0cd75-8408-48c1-b2ea-c2603bc203bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:11.444875 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f14b4c-e2fc-4fc9-98cb-45e87d305db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:11.445551 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f14b4c-e2fc-4fc9-98cb-45e87d305db5 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:11.445834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f14b4c-e2fc-4fc9-98cb-45e87d305db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:11.446109 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f14b4c-e2fc-4fc9-98cb-45e87d305db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.447033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f14b4c-e2fc-4fc9-98cb-45e87d305db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:11.448465 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f14b4c-e2fc-4fc9-98cb-45e87d305db5 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:11.451684 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0cd75-8408-48c1-b2ea-c2603bc203bf None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:11.455618 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a577c-4b95-4b70-9370-d98927a6f4a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:11.459757 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a577c-4b95-4b70-9370-d98927a6f4a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:11.459757 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a577c-4b95-4b70-9370-d98927a6f4a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:11.459757 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a577c-4b95-4b70-9370-d98927a6f4a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.459757 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9f2a577c-4b95-4b70-9370-d98927a6f4a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:11.460124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1361/2703] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:11.462894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f14b4c-e2fc-4fc9-98cb-45e87d305db5 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:11.472679 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f14b4c-e2fc-4fc9-98cb-45e87d305db5 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:11.477263 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dcaef-eda5-4871-baff-7f6f420d1af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:11.477916 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dcaef-eda5-4871-baff-7f6f420d1af5 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:11.478200 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dcaef-eda5-4871-baff-7f6f420d1af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:11.478477 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dcaef-eda5-4871-baff-7f6f420d1af5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.479226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06dcaef-eda5-4871-baff-7f6f420d1af5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:11.484033 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06dcaef-eda5-4871-baff-7f6f420d1af5 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:11.499590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1362/2704] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:11.504258 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dcaef-eda5-4871-baff-7f6f420d1af5 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:11.514400 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dcaef-eda5-4871-baff-7f6f420d1af5 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:11.520417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ccc33-9cef-4064-9508-50b60af30eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:11.522925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ccc33-9cef-4064-9508-50b60af30eaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:11.522925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ccc33-9cef-4064-9508-50b60af30eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:11.522925 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ccc33-9cef-4064-9508-50b60af30eaa admin admin] PATH_INFO: `/v3/projects/40e08dea949e4a27a3ebad1fab9359e3/users/d5453ab5c00c457488c472272ae055fa/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.539342 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a577c-4b95-4b70-9370-d98927a6f4a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:11.540943 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a577c-4b95-4b70-9370-d98927a6f4a7 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:11.542574 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ccc33-9cef-4064-9508-50b60af30eaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=40e08dea949e4a27a3ebad1fab9359e3, user_id=d5453ab5c00c457488c472272ae055fa, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:11.545755 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ccc33-9cef-4064-9508-50b60af30eaa admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:11.555549 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1343/2705] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:57:11.619462 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1363/2706] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:11 2022] PUT /identity/v3/projects/40e08dea949e4a27a3ebad1fab9359e3/users/d5453ab5c00c457488c472272ae055fa/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:11.630077 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1279702-244b-4368-aa36-1e13d67ff63a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:11.630371 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1279702-244b-4368-aa36-1e13d67ff63a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:11.630698 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1279702-244b-4368-aa36-1e13d67ff63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.660391 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a1279702-244b-4368-aa36-1e13d67ff63a None None] MFA Rules not processed for user `d5453ab5c00c457488c472272ae055fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:11.663002 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a1279702-244b-4368-aa36-1e13d67ff63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:11.729759 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1344/2707] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:11.735224 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ccc33-9cef-4064-9508-50b60af30eaa admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:11.780123 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:11.800789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ccc33-9cef-4064-9508-50b60af30eaa admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:11.805687 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c7e16-f434-46a8-970c-aff0a741c0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:11.806347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c7e16-f434-46a8-970c-aff0a741c0cb admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:11.806840 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c7e16-f434-46a8-970c-aff0a741c0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:11.807124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c7e16-f434-46a8-970c-aff0a741c0cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.807832 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c7e16-f434-46a8-970c-aff0a741c0cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:11.809741 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c7e16-f434-46a8-970c-aff0a741c0cb admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:11.996715 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1364/2708] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:11 2022] POST /identity/v3/projects => generated 385 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:11.996846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1279702-244b-4368-aa36-1e13d67ff63a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:11.996846 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1279702-244b-4368-aa36-1e13d67ff63a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:12.055000 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb25fe-8249-41f9-a38e-d0fae700e956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:12.055916 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb25fe-8249-41f9-a38e-d0fae700e956 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:12.056278 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb25fe-8249-41f9-a38e-d0fae700e956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:12.056584 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb25fe-8249-41f9-a38e-d0fae700e956 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:12.057382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb25fe-8249-41f9-a38e-d0fae700e956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:12.059522 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb25fe-8249-41f9-a38e-d0fae700e956 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:12.191683 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1345/2709] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:11 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:12.195562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c7e16-f434-46a8-970c-aff0a741c0cb admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:12.206514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c7e16-f434-46a8-970c-aff0a741c0cb admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:12.211063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871c4b1-c380-4c21-9c7e-1d7caf1350e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:12.211727 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871c4b1-c380-4c21-9c7e-1d7caf1350e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:12.212029 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871c4b1-c380-4c21-9c7e-1d7caf1350e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:12.212291 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871c4b1-c380-4c21-9c7e-1d7caf1350e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:12.213017 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871c4b1-c380-4c21-9c7e-1d7caf1350e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:12.214670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871c4b1-c380-4c21-9c7e-1d7caf1350e8 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:12.226167 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1365/2710] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:12.230186 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb25fe-8249-41f9-a38e-d0fae700e956 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:12.241065 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb25fe-8249-41f9-a38e-d0fae700e956 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:12.246043 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0d5f6-c933-4db3-9a72-41e0f583d937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:12.246833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0d5f6-c933-4db3-9a72-41e0f583d937 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:12.247181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0d5f6-c933-4db3-9a72-41e0f583d937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:12.247539 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0d5f6-c933-4db3-9a72-41e0f583d937 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:12.248422 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0d5f6-c933-4db3-9a72-41e0f583d937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:12.250380 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0d5f6-c933-4db3-9a72-41e0f583d937 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:12.263606 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1346/2711] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:12.264914 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871c4b1-c380-4c21-9c7e-1d7caf1350e8 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:12.275616 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871c4b1-c380-4c21-9c7e-1d7caf1350e8 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:12.280252 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ed2f3-7691-4b5c-91ff-4974122bea78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:12.281160 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ed2f3-7691-4b5c-91ff-4974122bea78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:12.281523 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ed2f3-7691-4b5c-91ff-4974122bea78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:12.282568 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ed2f3-7691-4b5c-91ff-4974122bea78 admin admin] PATH_INFO: `/v3/projects/4d1432435f1d4af08367ca57dcd6e484/users/9395c4c3576343fa8e55e16f02b42829/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:12.297669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ed2f3-7691-4b5c-91ff-4974122bea78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d1432435f1d4af08367ca57dcd6e484, user_id=9395c4c3576343fa8e55e16f02b42829, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:12.300349 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ed2f3-7691-4b5c-91ff-4974122bea78 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:12.370154 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1366/2712] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:12 2022] PUT /identity/v3/projects/4d1432435f1d4af08367ca57dcd6e484/users/9395c4c3576343fa8e55e16f02b42829/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:12.383720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4c51e-7228-46b7-b947-5e14be5f6bcd None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:12.384257 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4c51e-7228-46b7-b947-5e14be5f6bcd None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:12.385398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4c51e-7228-46b7-b947-5e14be5f6bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:12.416301 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-14d4c51e-7228-46b7-b947-5e14be5f6bcd None None] MFA Rules not processed for user `9395c4c3576343fa8e55e16f02b42829`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:12.417601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-14d4c51e-7228-46b7-b947-5e14be5f6bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:12.465939 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:12.485101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1347/2713] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:12.490083 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ed2f3-7691-4b5c-91ff-4974122bea78 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:12.553634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ed2f3-7691-4b5c-91ff-4974122bea78 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:12.558731 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77d3b8-2f43-462e-9f1c-b2b8aaffaba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:12.559384 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77d3b8-2f43-462e-9f1c-b2b8aaffaba8 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:12.559658 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77d3b8-2f43-462e-9f1c-b2b8aaffaba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:12.559933 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77d3b8-2f43-462e-9f1c-b2b8aaffaba8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:12.560630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77d3b8-2f43-462e-9f1c-b2b8aaffaba8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:12.563186 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77d3b8-2f43-462e-9f1c-b2b8aaffaba8 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:12.677391 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1367/2714] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:12 2022] POST /identity/v3/projects => generated 387 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:12.678507 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4c51e-7228-46b7-b947-5e14be5f6bcd None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:12.690067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4c51e-7228-46b7-b947-5e14be5f6bcd None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:12.740722 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:12.757007 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426ee22-3710-414b-bc4b-ba533e89bf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:12.757680 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426ee22-3710-414b-bc4b-ba533e89bf98 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:12.757962 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426ee22-3710-414b-bc4b-ba533e89bf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:12.758357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426ee22-3710-414b-bc4b-ba533e89bf98 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:12.759501 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426ee22-3710-414b-bc4b-ba533e89bf98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:12.761569 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426ee22-3710-414b-bc4b-ba533e89bf98 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:12.861643 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1348/2715] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:12 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:12.865647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77d3b8-2f43-462e-9f1c-b2b8aaffaba8 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:12.876201 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77d3b8-2f43-462e-9f1c-b2b8aaffaba8 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:12.881288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba03d0e-ec94-48e7-abb4-ca5c95d4b0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:12.882210 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba03d0e-ec94-48e7-abb4-ca5c95d4b0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:12.882598 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba03d0e-ec94-48e7-abb4-ca5c95d4b0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:12.882960 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba03d0e-ec94-48e7-abb4-ca5c95d4b0d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:12.883758 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba03d0e-ec94-48e7-abb4-ca5c95d4b0d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:12.885369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba03d0e-ec94-48e7-abb4-ca5c95d4b0d9 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:12.900531 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1368/2716] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:12.901862 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426ee22-3710-414b-bc4b-ba533e89bf98 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:12.913228 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426ee22-3710-414b-bc4b-ba533e89bf98 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:12.918381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6084a7-737d-410a-b4ea-82f23362f396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:12.919394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6084a7-737d-410a-b4ea-82f23362f396 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:12.927876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6084a7-737d-410a-b4ea-82f23362f396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:12.927876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6084a7-737d-410a-b4ea-82f23362f396 admin admin] PATH_INFO: `/v3/projects/b8abd1c4206e43bba04275f6f14ef7c5/users/10bc83eb7a46448ebc5740638e134250/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:12.936538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6084a7-737d-410a-b4ea-82f23362f396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8abd1c4206e43bba04275f6f14ef7c5, user_id=10bc83eb7a46448ebc5740638e134250, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:12.939390 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6084a7-737d-410a-b4ea-82f23362f396 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:13.003013 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1349/2717] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:12 2022] PUT /identity/v3/projects/b8abd1c4206e43bba04275f6f14ef7c5/users/10bc83eb7a46448ebc5740638e134250/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:13.010895 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba03d0e-ec94-48e7-abb4-ca5c95d4b0d9 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:13.021507 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba03d0e-ec94-48e7-abb4-ca5c95d4b0d9 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:13.090415 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6969614-aac6-436d-8e6f-7f25542a13cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:13.090715 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6969614-aac6-436d-8e6f-7f25542a13cd admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:13.091006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6969614-aac6-436d-8e6f-7f25542a13cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:13.091354 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6969614-aac6-436d-8e6f-7f25542a13cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:13.092022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6969614-aac6-436d-8e6f-7f25542a13cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:13.093737 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6969614-aac6-436d-8e6f-7f25542a13cd admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:13.108549 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1369/2718] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:13.109799 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6084a7-737d-410a-b4ea-82f23362f396 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:13.121069 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6084a7-737d-410a-b4ea-82f23362f396 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:13.126317 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4591c04-9a52-4931-9047-fc4d760d85a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:13.127251 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4591c04-9a52-4931-9047-fc4d760d85a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:13.127539 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4591c04-9a52-4931-9047-fc4d760d85a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:13.127844 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4591c04-9a52-4931-9047-fc4d760d85a0 admin admin] PATH_INFO: `/v3/projects/b8abd1c4206e43bba04275f6f14ef7c5/users/10bc83eb7a46448ebc5740638e134250/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:13.129834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4591c04-9a52-4931-9047-fc4d760d85a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8abd1c4206e43bba04275f6f14ef7c5, user_id=10bc83eb7a46448ebc5740638e134250, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:13.132538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4591c04-9a52-4931-9047-fc4d760d85a0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:13.144001 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1350/2719] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:13 2022] PUT /identity/v3/projects/b8abd1c4206e43bba04275f6f14ef7c5/users/10bc83eb7a46448ebc5740638e134250/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:13.154147 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d36f93-fd23-49f6-bbdb-c42a1fe86d2c None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:13.154442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d36f93-fd23-49f6-bbdb-c42a1fe86d2c None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:13.154764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d36f93-fd23-49f6-bbdb-c42a1fe86d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:13.181496 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-36d36f93-fd23-49f6-bbdb-c42a1fe86d2c None None] MFA Rules not processed for user `10bc83eb7a46448ebc5740638e134250`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:13.182960 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-36d36f93-fd23-49f6-bbdb-c42a1fe86d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:13.255429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1370/2720] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:13.267336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4591c04-9a52-4931-9047-fc4d760d85a0 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:13.323288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4591c04-9a52-4931-9047-fc4d760d85a0 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:13.327006 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fe48a-4824-4677-9f86-e921b050dbf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:13.327659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fe48a-4824-4677-9f86-e921b050dbf5 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:13.327968 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fe48a-4824-4677-9f86-e921b050dbf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:13.328393 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fe48a-4824-4677-9f86-e921b050dbf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:13.338350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042fe48a-4824-4677-9f86-e921b050dbf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:13.339778 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042fe48a-4824-4677-9f86-e921b050dbf5 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:13.350492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1351/2721] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:13.709599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:13.995935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:15.251306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d36f93-fd23-49f6-bbdb-c42a1fe86d2c None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:15.260780 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d36f93-fd23-49f6-bbdb-c42a1fe86d2c None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:15.267810 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290d115-f359-4ab8-899f-3bb16fd957bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:15.268518 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290d115-f359-4ab8-899f-3bb16fd957bb service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:15.268806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290d115-f359-4ab8-899f-3bb16fd957bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:15.269529 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290d115-f359-4ab8-899f-3bb16fd957bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:15.330282 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290d115-f359-4ab8-899f-3bb16fd957bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:15.335364 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290d115-f359-4ab8-899f-3bb16fd957bb service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:15.340580 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1371/2722] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:15.716603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:16.135511 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fe48a-4824-4677-9f86-e921b050dbf5 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:16.161701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:16.202405 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fe48a-4824-4677-9f86-e921b050dbf5 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:16.207987 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465c330-90f5-459d-a634-19580ad7506c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:16.217152 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465c330-90f5-459d-a634-19580ad7506c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:16.217152 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465c330-90f5-459d-a634-19580ad7506c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:16.217152 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465c330-90f5-459d-a634-19580ad7506c admin admin] PATH_INFO: `/v3/users/53e27bd783c74442a72aa93ce00bedf7` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:16.217152 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4465c330-90f5-459d-a634-19580ad7506c admin admin] RBAC: Authorizing `identity:delete_user(user_id=53e27bd783c74442a72aa93ce00bedf7)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:16.217152 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4465c330-90f5-459d-a634-19580ad7506c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:16.485445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4465c330-90f5-459d-a634-19580ad7506c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53e27bd783c74442a72aa93ce00bedf7'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:16.492176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4465c330-90f5-459d-a634-19580ad7506c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53e27bd783c74442a72aa93ce00bedf7'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:16.492969 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4465c330-90f5-459d-a634-19580ad7506c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53e27bd783c74442a72aa93ce00bedf7'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:16.493758 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4465c330-90f5-459d-a634-19580ad7506c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53e27bd783c74442a72aa93ce00bedf7'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:16.505353 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4465c330-90f5-459d-a634-19580ad7506c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53e27bd783c74442a72aa93ce00bedf7'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:16.640624 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1352/2723] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:16 2022] DELETE /identity/v3/users/53e27bd783c74442a72aa93ce00bedf7 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:16.646041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc1da1-fabb-4f7c-8d13-12f319bfc7e8 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:16.646522 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc1da1-fabb-4f7c-8d13-12f319bfc7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:16.646691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc1da1-fabb-4f7c-8d13-12f319bfc7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:16.664604 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-86fc1da1-fabb-4f7c-8d13-12f319bfc7e8 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:16.665857 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-86fc1da1-fabb-4f7c-8d13-12f319bfc7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:16.724982 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1372/2724] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:16.735246 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465c330-90f5-459d-a634-19580ad7506c admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:16.736741 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4465c330-90f5-459d-a634-19580ad7506c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:16.816322 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465c330-90f5-459d-a634-19580ad7506c admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:16.819965 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bede6-f196-40b6-8cf9-8cf402cada6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:16.820616 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bede6-f196-40b6-8cf9-8cf402cada6b service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:16.820899 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bede6-f196-40b6-8cf9-8cf402cada6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:16.821178 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bede6-f196-40b6-8cf9-8cf402cada6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:16.830757 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69bede6-f196-40b6-8cf9-8cf402cada6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:16.832189 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69bede6-f196-40b6-8cf9-8cf402cada6b service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:16.841582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1353/2725] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:17.318689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc1da1-fabb-4f7c-8d13-12f319bfc7e8 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:17.320186 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-86fc1da1-fabb-4f7c-8d13-12f319bfc7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:17.333241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc1da1-fabb-4f7c-8d13-12f319bfc7e8 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:17.337898 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2bbd6-b59e-4ab6-9856-315994fecaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:17.338808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2bbd6-b59e-4ab6-9856-315994fecaa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:17.339087 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2bbd6-b59e-4ab6-9856-315994fecaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:17.339356 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2bbd6-b59e-4ab6-9856-315994fecaa2 admin admin] PATH_INFO: `/v3/projects/72359b7d3cdf4a9a8bec00cbf1e1aa17` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:17.341124 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc2bbd6-b59e-4ab6-9856-315994fecaa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72359b7d3cdf4a9a8bec00cbf1e1aa17)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:17.343173 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc2bbd6-b59e-4ab6-9856-315994fecaa2 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:17.503382 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fdc2bbd6-b59e-4ab6-9856-315994fecaa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72359b7d3cdf4a9a8bec00cbf1e1aa17', 'request_id': 'req-fdc2bbd6-b59e-4ab6-9856-315994fecaa2'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:17.514325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fdc2bbd6-b59e-4ab6-9856-315994fecaa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72359b7d3cdf4a9a8bec00cbf1e1aa17', 'request_id': 'req-fdc2bbd6-b59e-4ab6-9856-315994fecaa2'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:17.619263 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fdc2bbd6-b59e-4ab6-9856-315994fecaa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72359b7d3cdf4a9a8bec00cbf1e1aa17', 'request_id': 'req-fdc2bbd6-b59e-4ab6-9856-315994fecaa2'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:17.630851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fdc2bbd6-b59e-4ab6-9856-315994fecaa2 admin admin] The token cache is being invalidate because project 72359b7d3cdf4a9a8bec00cbf1e1aa17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:17.639193 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fdc2bbd6-b59e-4ab6-9856-315994fecaa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:17.639193 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-fdc2bbd6-b59e-4ab6-9856-315994fecaa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:17.639193 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1373/2726] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:17 2022] DELETE /identity/v3/projects/72359b7d3cdf4a9a8bec00cbf1e1aa17 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:17.639970 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bede6-f196-40b6-8cf9-8cf402cada6b service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:17.647351 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bede6-f196-40b6-8cf9-8cf402cada6b service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:17.691003 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:17.705768 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc389e3-c3d7-44f1-821e-f184e9359265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:17.706677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc389e3-c3d7-44f1-821e-f184e9359265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:17.706967 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc389e3-c3d7-44f1-821e-f184e9359265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:17.707244 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc389e3-c3d7-44f1-821e-f184e9359265 admin admin] PATH_INFO: `/v3/users/ebeb8f549a8648c4ac513aa66e761a1c` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:17.708472 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc389e3-c3d7-44f1-821e-f184e9359265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebeb8f549a8648c4ac513aa66e761a1c)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:17.710608 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc389e3-c3d7-44f1-821e-f184e9359265 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:17.939586 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9cc389e3-c3d7-44f1-821e-f184e9359265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebeb8f549a8648c4ac513aa66e761a1c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:17.946314 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9cc389e3-c3d7-44f1-821e-f184e9359265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebeb8f549a8648c4ac513aa66e761a1c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:17.947234 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9cc389e3-c3d7-44f1-821e-f184e9359265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebeb8f549a8648c4ac513aa66e761a1c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:17.947916 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9cc389e3-c3d7-44f1-821e-f184e9359265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebeb8f549a8648c4ac513aa66e761a1c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:17.962246 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9cc389e3-c3d7-44f1-821e-f184e9359265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebeb8f549a8648c4ac513aa66e761a1c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:18.041875 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1354/2727] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:17 2022] DELETE /identity/v3/users/ebeb8f549a8648c4ac513aa66e761a1c => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:18.455339 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2bbd6-b59e-4ab6-9856-315994fecaa2 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:18.457215 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-fdc2bbd6-b59e-4ab6-9856-315994fecaa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:18.502338 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2bbd6-b59e-4ab6-9856-315994fecaa2 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:18.513887 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:18.555755 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83f772-b8ea-4298-a8c6-a985c22e1904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:18.557355 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83f772-b8ea-4298-a8c6-a985c22e1904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:18.557631 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83f772-b8ea-4298-a8c6-a985c22e1904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:18.557916 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83f772-b8ea-4298-a8c6-a985c22e1904 admin admin] PATH_INFO: `/v3/projects/5b6dc31874b2442f9029bf18b21b30a9` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:18.559301 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83f772-b8ea-4298-a8c6-a985c22e1904 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b6dc31874b2442f9029bf18b21b30a9)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:18.561267 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83f772-b8ea-4298-a8c6-a985c22e1904 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:18.705681 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-cb83f772-b8ea-4298-a8c6-a985c22e1904 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b6dc31874b2442f9029bf18b21b30a9', 'request_id': 'req-cb83f772-b8ea-4298-a8c6-a985c22e1904'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:18.712274 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-cb83f772-b8ea-4298-a8c6-a985c22e1904 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b6dc31874b2442f9029bf18b21b30a9', 'request_id': 'req-cb83f772-b8ea-4298-a8c6-a985c22e1904'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:18.801732 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-cb83f772-b8ea-4298-a8c6-a985c22e1904 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b6dc31874b2442f9029bf18b21b30a9', 'request_id': 'req-cb83f772-b8ea-4298-a8c6-a985c22e1904'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:18.816778 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-cb83f772-b8ea-4298-a8c6-a985c22e1904 admin admin] The token cache is being invalidate because project 5b6dc31874b2442f9029bf18b21b30a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:18.816778 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-cb83f772-b8ea-4298-a8c6-a985c22e1904 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:18.816778 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-cb83f772-b8ea-4298-a8c6-a985c22e1904 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:18.816778 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1374/2728] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:18 2022] DELETE /identity/v3/projects/5b6dc31874b2442f9029bf18b21b30a9 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:18.822259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c124ed7-a078-49e0-a5bc-164544968991 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:18.822784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c124ed7-a078-49e0-a5bc-164544968991 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:18.822882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c124ed7-a078-49e0-a5bc-164544968991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:18.841497 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-9c124ed7-a078-49e0-a5bc-164544968991 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:18.842765 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9c124ed7-a078-49e0-a5bc-164544968991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:18.904504 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1355/2729] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:18.913015 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47429302-c5b5-497f-8c47-aead61a2ae7c None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:18.913312 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47429302-c5b5-497f-8c47-aead61a2ae7c None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:18.913592 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47429302-c5b5-497f-8c47-aead61a2ae7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:18.934144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-47429302-c5b5-497f-8c47-aead61a2ae7c None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:18.935725 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-47429302-c5b5-497f-8c47-aead61a2ae7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:18.944760 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1375/2730] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:18.949390 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c124ed7-a078-49e0-a5bc-164544968991 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:18.960537 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c124ed7-a078-49e0-a5bc-164544968991 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:18.965265 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d7297-4571-4557-b238-aa1b43c24147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:18.965928 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d7297-4571-4557-b238-aa1b43c24147 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:18.966211 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d7297-4571-4557-b238-aa1b43c24147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:18.966518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d7297-4571-4557-b238-aa1b43c24147 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:18.968045 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d7297-4571-4557-b238-aa1b43c24147 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:57:18.968045 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d7297-4571-4557-b238-aa1b43c24147 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:18.969182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d7297-4571-4557-b238-aa1b43c24147 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:18.982125 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1356/2731] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:57:18.986388 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47429302-c5b5-497f-8c47-aead61a2ae7c None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:18.996591 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47429302-c5b5-497f-8c47-aead61a2ae7c None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:19.002135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0370fd-0f1e-4d63-8369-4104e2755cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:19.002811 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0370fd-0f1e-4d63-8369-4104e2755cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:19.003197 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0370fd-0f1e-4d63-8369-4104e2755cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:19.003424 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0370fd-0f1e-4d63-8369-4104e2755cb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:19.004112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0370fd-0f1e-4d63-8369-4104e2755cb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:19.006161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0370fd-0f1e-4d63-8369-4104e2755cb7 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:19.137522 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1376/2732] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:18 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:19.138591 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d7297-4571-4557-b238-aa1b43c24147 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:19.149762 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d7297-4571-4557-b238-aa1b43c24147 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:19.201249 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:19.215784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7b9b8-30f1-4a8f-8176-85f053263bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:19.216801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7b9b8-30f1-4a8f-8176-85f053263bda admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:19.217133 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7b9b8-30f1-4a8f-8176-85f053263bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:19.217552 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7b9b8-30f1-4a8f-8176-85f053263bda admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:19.218366 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7b9b8-30f1-4a8f-8176-85f053263bda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:19.220392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7b9b8-30f1-4a8f-8176-85f053263bda admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:19.311650 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1357/2733] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:19 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:19.316198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0370fd-0f1e-4d63-8369-4104e2755cb7 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:19.326679 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0370fd-0f1e-4d63-8369-4104e2755cb7 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:19.332166 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d446fc2-f980-406b-940d-7644928b6de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:19.339517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d446fc2-f980-406b-940d-7644928b6de1 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:19.339517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d446fc2-f980-406b-940d-7644928b6de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:19.339517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d446fc2-f980-406b-940d-7644928b6de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:19.339517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d446fc2-f980-406b-940d-7644928b6de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:19.339517 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d446fc2-f980-406b-940d-7644928b6de1 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:19.347371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1377/2734] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:19.351582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7b9b8-30f1-4a8f-8176-85f053263bda admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:19.362192 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7b9b8-30f1-4a8f-8176-85f053263bda admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:19.366912 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a28d1-454e-4942-bcd2-2f91882f9fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:19.367563 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a28d1-454e-4942-bcd2-2f91882f9fcb admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:19.367843 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a28d1-454e-4942-bcd2-2f91882f9fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:19.368116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a28d1-454e-4942-bcd2-2f91882f9fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:19.368838 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a28d1-454e-4942-bcd2-2f91882f9fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:19.370419 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a28d1-454e-4942-bcd2-2f91882f9fcb admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:19.382444 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1358/2735] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:19.386495 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d446fc2-f980-406b-940d-7644928b6de1 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:19.397181 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d446fc2-f980-406b-940d-7644928b6de1 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:19.401741 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c779f-262f-4081-940a-ca5a157fbae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:19.402651 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c779f-262f-4081-940a-ca5a157fbae7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:19.402937 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c779f-262f-4081-940a-ca5a157fbae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:19.403225 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c779f-262f-4081-940a-ca5a157fbae7 admin admin] PATH_INFO: `/v3/projects/51d6d52f57ea42b299be69ae9f80e78e/users/9d3d9f50a04f4cf9aa7efebde17bc3ac/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:19.417729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4c779f-262f-4081-940a-ca5a157fbae7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51d6d52f57ea42b299be69ae9f80e78e, user_id=9d3d9f50a04f4cf9aa7efebde17bc3ac, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:19.420417 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4c779f-262f-4081-940a-ca5a157fbae7 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:19.499813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1378/2736] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:19 2022] PUT /identity/v3/projects/51d6d52f57ea42b299be69ae9f80e78e/users/9d3d9f50a04f4cf9aa7efebde17bc3ac/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:19.511686 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e792cb0-54f8-4af9-a2f7-0fbdc287ec3e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:19.511983 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e792cb0-54f8-4af9-a2f7-0fbdc287ec3e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:19.512334 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e792cb0-54f8-4af9-a2f7-0fbdc287ec3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:19.539562 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-9e792cb0-54f8-4af9-a2f7-0fbdc287ec3e None None] MFA Rules not processed for user `9d3d9f50a04f4cf9aa7efebde17bc3ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:19.540780 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9e792cb0-54f8-4af9-a2f7-0fbdc287ec3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:19.607677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1359/2737] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:19.612859 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c779f-262f-4081-940a-ca5a157fbae7 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:19.661308 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:19.674716 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c779f-262f-4081-940a-ca5a157fbae7 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:19.679319 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6ea36-9b7e-4429-a62b-4e19223ea827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:19.679970 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6ea36-9b7e-4429-a62b-4e19223ea827 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:19.680254 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6ea36-9b7e-4429-a62b-4e19223ea827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:19.680527 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6ea36-9b7e-4429-a62b-4e19223ea827 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:19.681208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b6ea36-9b7e-4429-a62b-4e19223ea827 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:19.683226 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b6ea36-9b7e-4429-a62b-4e19223ea827 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:19.815677 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1379/2738] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:19 2022] POST /identity/v3/projects => generated 381 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:19.819599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e792cb0-54f8-4af9-a2f7-0fbdc287ec3e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:19.830804 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e792cb0-54f8-4af9-a2f7-0fbdc287ec3e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:19.894568 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3972ebf-594d-4a09-996f-c7f8a7e4785d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:19.895221 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3972ebf-594d-4a09-996f-c7f8a7e4785d admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:19.895517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3972ebf-594d-4a09-996f-c7f8a7e4785d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:19.895834 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3972ebf-594d-4a09-996f-c7f8a7e4785d admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:19.898980 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3972ebf-594d-4a09-996f-c7f8a7e4785d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:19.898980 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3972ebf-594d-4a09-996f-c7f8a7e4785d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:19.945038 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6ea36-9b7e-4429-a62b-4e19223ea827 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:19.947526 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-41b6ea36-9b7e-4429-a62b-4e19223ea827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:19.998339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1360/2739] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:19 2022] POST /identity/v3/users => generated 291 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:20.002348 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3972ebf-594d-4a09-996f-c7f8a7e4785d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:20.014287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3972ebf-594d-4a09-996f-c7f8a7e4785d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:20.021148 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62935da7-b4fa-41a5-923b-04e7b2bef93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:20.021866 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62935da7-b4fa-41a5-923b-04e7b2bef93e admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:20.022245 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62935da7-b4fa-41a5-923b-04e7b2bef93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:20.022650 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62935da7-b4fa-41a5-923b-04e7b2bef93e admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:20.023476 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62935da7-b4fa-41a5-923b-04e7b2bef93e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:20.025751 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62935da7-b4fa-41a5-923b-04e7b2bef93e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:20.038231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1361/2740] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:20.041784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62935da7-b4fa-41a5-923b-04e7b2bef93e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:20.052605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62935da7-b4fa-41a5-923b-04e7b2bef93e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:20.058210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3e3c9-ea5f-4acd-8f20-e1d72f5e46ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:20.058923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3e3c9-ea5f-4acd-8f20-e1d72f5e46ae admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:20.059205 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3e3c9-ea5f-4acd-8f20-e1d72f5e46ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:20.059479 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3e3c9-ea5f-4acd-8f20-e1d72f5e46ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:20.060203 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3e3c9-ea5f-4acd-8f20-e1d72f5e46ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:20.061825 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3e3c9-ea5f-4acd-8f20-e1d72f5e46ae admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:20.068006 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6ea36-9b7e-4429-a62b-4e19223ea827 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:20.075007 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd1b00-34d9-4ff2-9603-d215f7f11fd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '161c5ed1e47e450290a1f428ee100b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:20.075990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd1b00-34d9-4ff2-9603-d215f7f11fd9 service nova] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:20.075990 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd1b00-34d9-4ff2-9603-d215f7f11fd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:20.076296 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd1b00-34d9-4ff2-9603-d215f7f11fd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:20.076561 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1362/2741] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:20.077410 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3e3c9-ea5f-4acd-8f20-e1d72f5e46ae admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:20.078037 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-24fd1b00-34d9-4ff2-9603-d215f7f11fd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:20.087565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3e3c9-ea5f-4acd-8f20-e1d72f5e46ae admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:20.091412 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd1b00-34d9-4ff2-9603-d215f7f11fd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:20.095500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0237fcf-16e6-4226-bad7-0503b31e7482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:20.096113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd1b00-34d9-4ff2-9603-d215f7f11fd9 service nova] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:20.096229 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0237fcf-16e6-4226-bad7-0503b31e7482 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:20.096229 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0237fcf-16e6-4226-bad7-0503b31e7482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:20.096229 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0237fcf-16e6-4226-bad7-0503b31e7482 admin admin] PATH_INFO: `/v3/projects/2f2144028cfc48a28f0799edfe60354a/users/c46b4678cabe42d5b504745d4feb7f14/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:20.104501 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1380/2742] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:20.112901 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0237fcf-16e6-4226-bad7-0503b31e7482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f2144028cfc48a28f0799edfe60354a, user_id=c46b4678cabe42d5b504745d4feb7f14, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:20.115641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0237fcf-16e6-4226-bad7-0503b31e7482 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:20.191067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1363/2743] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:20 2022] PUT /identity/v3/projects/2f2144028cfc48a28f0799edfe60354a/users/c46b4678cabe42d5b504745d4feb7f14/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:20.201935 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c0dc6-60a6-45a7-98dd-d64c1c3fc16f None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:20.202304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c0dc6-60a6-45a7-98dd-d64c1c3fc16f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:20.202691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c0dc6-60a6-45a7-98dd-d64c1c3fc16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:20.232860 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-4f3c0dc6-60a6-45a7-98dd-d64c1c3fc16f None None] MFA Rules not processed for user `c46b4678cabe42d5b504745d4feb7f14`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:20.234117 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-4f3c0dc6-60a6-45a7-98dd-d64c1c3fc16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:20.306201 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1381/2744] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:20.311156 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0237fcf-16e6-4226-bad7-0503b31e7482 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:20.316280 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:20.375720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0237fcf-16e6-4226-bad7-0503b31e7482 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:20.380784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e404f-986a-4c1e-80ee-9ba48785ea22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:20.381448 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e404f-986a-4c1e-80ee-9ba48785ea22 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:20.381735 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e404f-986a-4c1e-80ee-9ba48785ea22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:20.382011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e404f-986a-4c1e-80ee-9ba48785ea22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:20.382792 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e404f-986a-4c1e-80ee-9ba48785ea22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:20.384784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e404f-986a-4c1e-80ee-9ba48785ea22 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:20.468606 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c0dc6-60a6-45a7-98dd-d64c1c3fc16f None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:20.470793 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-4f3c0dc6-60a6-45a7-98dd-d64c1c3fc16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:20.497534 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1364/2745] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:20 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:20.501446 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e404f-986a-4c1e-80ee-9ba48785ea22 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:20.548096 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c0dc6-60a6-45a7-98dd-d64c1c3fc16f None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:20.571421 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e404f-986a-4c1e-80ee-9ba48785ea22 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:20.576263 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786e011-eae4-478d-8a9e-fd8b01b7c6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:20.576935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786e011-eae4-478d-8a9e-fd8b01b7c6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:20.577317 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786e011-eae4-478d-8a9e-fd8b01b7c6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:20.577604 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786e011-eae4-478d-8a9e-fd8b01b7c6e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:20.578329 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786e011-eae4-478d-8a9e-fd8b01b7c6e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:20.580725 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786e011-eae4-478d-8a9e-fd8b01b7c6e2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:20.598389 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43397681-d51a-4d5a-ac58-2f38c558c912 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:20.599072 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43397681-d51a-4d5a-ac58-2f38c558c912 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:20.599359 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43397681-d51a-4d5a-ac58-2f38c558c912 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:20.599641 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43397681-d51a-4d5a-ac58-2f38c558c912 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:20.601468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-43397681-d51a-4d5a-ac58-2f38c558c912 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:20.661026 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1365/2746] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:20 2022] POST /identity/v3/users => generated 299 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:20.661259 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43397681-d51a-4d5a-ac58-2f38c558c912 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:20.662839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43397681-d51a-4d5a-ac58-2f38c558c912 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:20.664842 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786e011-eae4-478d-8a9e-fd8b01b7c6e2 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:20.672029 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1382/2747] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:57:20.675497 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786e011-eae4-478d-8a9e-fd8b01b7c6e2 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:20.680705 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac86946-d80e-4160-ac64-49d5428a7116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:20.680994 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac86946-d80e-4160-ac64-49d5428a7116 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:20.681278 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac86946-d80e-4160-ac64-49d5428a7116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:20.681554 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac86946-d80e-4160-ac64-49d5428a7116 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:20.682283 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac86946-d80e-4160-ac64-49d5428a7116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:20.683917 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac86946-d80e-4160-ac64-49d5428a7116 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:20.694651 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1366/2748] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:20.698950 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43397681-d51a-4d5a-ac58-2f38c558c912 service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:20.710239 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43397681-d51a-4d5a-ac58-2f38c558c912 service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:20.714874 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233460a-828a-45ba-9dfc-dbdb414fd01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:20.715752 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233460a-828a-45ba-9dfc-dbdb414fd01d admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:20.716030 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233460a-828a-45ba-9dfc-dbdb414fd01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:20.716304 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233460a-828a-45ba-9dfc-dbdb414fd01d admin admin] PATH_INFO: `/v3/projects/a89f70db6f184174842f7b50b62c6e42/users/fb0de0396503407490b97acdc7c1faad/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:20.732359 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233460a-828a-45ba-9dfc-dbdb414fd01d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a89f70db6f184174842f7b50b62c6e42, user_id=fb0de0396503407490b97acdc7c1faad, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:20.735952 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233460a-828a-45ba-9dfc-dbdb414fd01d admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:20.843501 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1383/2749] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:20 2022] PUT /identity/v3/projects/a89f70db6f184174842f7b50b62c6e42/users/fb0de0396503407490b97acdc7c1faad/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:20.848969 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac86946-d80e-4160-ac64-49d5428a7116 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:20.867048 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac86946-d80e-4160-ac64-49d5428a7116 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:20.941242 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3124c9a-75d9-4554-b079-6fadd9b7e782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:20.941921 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3124c9a-75d9-4554-b079-6fadd9b7e782 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:20.942207 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3124c9a-75d9-4554-b079-6fadd9b7e782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:20.942482 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3124c9a-75d9-4554-b079-6fadd9b7e782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:20.943334 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3124c9a-75d9-4554-b079-6fadd9b7e782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:20.945051 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3124c9a-75d9-4554-b079-6fadd9b7e782 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:20.959581 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1367/2750] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:20.960944 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233460a-828a-45ba-9dfc-dbdb414fd01d admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:20.972362 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233460a-828a-45ba-9dfc-dbdb414fd01d admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:20.977040 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d5768-07a4-4b99-ad3f-446331a00eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:20.977955 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d5768-07a4-4b99-ad3f-446331a00eb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:20.978238 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d5768-07a4-4b99-ad3f-446331a00eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:20.979248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d5768-07a4-4b99-ad3f-446331a00eb7 admin admin] PATH_INFO: `/v3/projects/a89f70db6f184174842f7b50b62c6e42/users/fb0de0396503407490b97acdc7c1faad/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:20.980806 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d5768-07a4-4b99-ad3f-446331a00eb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a89f70db6f184174842f7b50b62c6e42, user_id=fb0de0396503407490b97acdc7c1faad, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:20.983742 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d5768-07a4-4b99-ad3f-446331a00eb7 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:20.994350 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1384/2751] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:20 2022] PUT /identity/v3/projects/a89f70db6f184174842f7b50b62c6e42/users/fb0de0396503407490b97acdc7c1faad/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:21.004501 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cfdf6-4984-4cff-9c4d-cfedc609a4e6 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:21.004792 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cfdf6-4984-4cff-9c4d-cfedc609a4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:21.005071 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cfdf6-4984-4cff-9c4d-cfedc609a4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:21.035714 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-6e7cfdf6-4984-4cff-9c4d-cfedc609a4e6 None None] MFA Rules not processed for user `fb0de0396503407490b97acdc7c1faad`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:21.035714 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6e7cfdf6-4984-4cff-9c4d-cfedc609a4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:21.109166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1368/2752] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:21.120887 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d5768-07a4-4b99-ad3f-446331a00eb7 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:21.122874 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c60d5768-07a4-4b99-ad3f-446331a00eb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:21.178844 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:21.193021 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d5768-07a4-4b99-ad3f-446331a00eb7 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:21.197691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad56ec-681e-42ef-92ca-e776b4e02a5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:21.198344 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad56ec-681e-42ef-92ca-e776b4e02a5b service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:21.198694 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad56ec-681e-42ef-92ca-e776b4e02a5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:21.198921 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad56ec-681e-42ef-92ca-e776b4e02a5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:21.258683 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad56ec-681e-42ef-92ca-e776b4e02a5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:21.260046 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad56ec-681e-42ef-92ca-e776b4e02a5b service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:21.269399 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1385/2753] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:21.483230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:22.185054 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:23.455624 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cfdf6-4984-4cff-9c4d-cfedc609a4e6 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:23.467181 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cfdf6-4984-4cff-9c4d-cfedc609a4e6 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:23.471005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff998da-6650-450b-9c34-8d6c97c1c1ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:23.471680 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff998da-6650-450b-9c34-8d6c97c1c1ce service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:23.471962 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff998da-6650-450b-9c34-8d6c97c1c1ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:23.472236 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff998da-6650-450b-9c34-8d6c97c1c1ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:23.535333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff998da-6650-450b-9c34-8d6c97c1c1ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:23.541190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff998da-6650-450b-9c34-8d6c97c1c1ce service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:23.546555 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1369/2754] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:24.949669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad56ec-681e-42ef-92ca-e776b4e02a5b service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:24.960325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad56ec-681e-42ef-92ca-e776b4e02a5b service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:24.963989 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eaa6b9-c70e-4270-967b-b2c561aab9e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:24.965624 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eaa6b9-c70e-4270-967b-b2c561aab9e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:24.965624 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eaa6b9-c70e-4270-967b-b2c561aab9e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:24.965624 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eaa6b9-c70e-4270-967b-b2c561aab9e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:24.975096 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eaa6b9-c70e-4270-967b-b2c561aab9e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:24.976593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eaa6b9-c70e-4270-967b-b2c561aab9e0 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:24.986303 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1386/2755] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:24 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:25.600709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff998da-6650-450b-9c34-8d6c97c1c1ce service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:25.602264 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4ff998da-6650-450b-9c34-8d6c97c1c1ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:25.667620 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff998da-6650-450b-9c34-8d6c97c1c1ce service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:25.672201 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b8518-4e1f-49dc-a484-5391d22e116a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:25.673070 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b8518-4e1f-49dc-a484-5391d22e116a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:25.673359 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b8518-4e1f-49dc-a484-5391d22e116a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:25.673639 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b8518-4e1f-49dc-a484-5391d22e116a admin admin] PATH_INFO: `/v3/users/d5453ab5c00c457488c472272ae055fa` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:25.674868 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b8518-4e1f-49dc-a484-5391d22e116a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5453ab5c00c457488c472272ae055fa)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:25.676810 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b8518-4e1f-49dc-a484-5391d22e116a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:25.807925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:25.958098 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-156b8518-4e1f-49dc-a484-5391d22e116a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5453ab5c00c457488c472272ae055fa'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:25.964949 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-156b8518-4e1f-49dc-a484-5391d22e116a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5453ab5c00c457488c472272ae055fa'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:25.965908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-156b8518-4e1f-49dc-a484-5391d22e116a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5453ab5c00c457488c472272ae055fa'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:25.966869 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-156b8518-4e1f-49dc-a484-5391d22e116a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5453ab5c00c457488c472272ae055fa'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:25.978881 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-156b8518-4e1f-49dc-a484-5391d22e116a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5453ab5c00c457488c472272ae055fa'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:26.057487 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1370/2756] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:25 2022] DELETE /identity/v3/users/d5453ab5c00c457488c472272ae055fa => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:26.062026 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd40c61-b93a-42c1-88d4-33867aaaecf5 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:26.062499 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd40c61-b93a-42c1-88d4-33867aaaecf5 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:26.062619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd40c61-b93a-42c1-88d4-33867aaaecf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:26.080245 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-4cd40c61-b93a-42c1-88d4-33867aaaecf5 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:26.082158 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-4cd40c61-b93a-42c1-88d4-33867aaaecf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:26.138995 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1387/2757] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:26.149237 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b8518-4e1f-49dc-a484-5391d22e116a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:26.150735 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-156b8518-4e1f-49dc-a484-5391d22e116a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:26.207611 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b8518-4e1f-49dc-a484-5391d22e116a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:26.211247 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f9590-5798-42f2-9e4a-07f6067e5ef8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:26.211898 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f9590-5798-42f2-9e4a-07f6067e5ef8 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:26.212182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f9590-5798-42f2-9e4a-07f6067e5ef8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:26.212461 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f9590-5798-42f2-9e4a-07f6067e5ef8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:26.221425 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f9590-5798-42f2-9e4a-07f6067e5ef8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:26.223093 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f9590-5798-42f2-9e4a-07f6067e5ef8 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:26.233118 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1371/2758] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:26.770640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd40c61-b93a-42c1-88d4-33867aaaecf5 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:26.773035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-4cd40c61-b93a-42c1-88d4-33867aaaecf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:26.786588 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd40c61-b93a-42c1-88d4-33867aaaecf5 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:26.802862 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd83bc8-93dc-4e0b-a05d-fa03b2312272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:26.804346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd83bc8-93dc-4e0b-a05d-fa03b2312272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:26.804633 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd83bc8-93dc-4e0b-a05d-fa03b2312272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:26.804913 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd83bc8-93dc-4e0b-a05d-fa03b2312272 admin admin] PATH_INFO: `/v3/projects/40e08dea949e4a27a3ebad1fab9359e3` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:26.806437 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd83bc8-93dc-4e0b-a05d-fa03b2312272 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40e08dea949e4a27a3ebad1fab9359e3)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:26.808637 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd83bc8-93dc-4e0b-a05d-fa03b2312272 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:26.946851 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3bd83bc8-93dc-4e0b-a05d-fa03b2312272 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40e08dea949e4a27a3ebad1fab9359e3', 'request_id': 'req-3bd83bc8-93dc-4e0b-a05d-fa03b2312272'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:26.953523 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3bd83bc8-93dc-4e0b-a05d-fa03b2312272 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40e08dea949e4a27a3ebad1fab9359e3', 'request_id': 'req-3bd83bc8-93dc-4e0b-a05d-fa03b2312272'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:27.041968 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3bd83bc8-93dc-4e0b-a05d-fa03b2312272 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40e08dea949e4a27a3ebad1fab9359e3', 'request_id': 'req-3bd83bc8-93dc-4e0b-a05d-fa03b2312272'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:27.053411 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3bd83bc8-93dc-4e0b-a05d-fa03b2312272 admin admin] The token cache is being invalidate because project 40e08dea949e4a27a3ebad1fab9359e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:27.053702 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3bd83bc8-93dc-4e0b-a05d-fa03b2312272 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:27.054344 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3bd83bc8-93dc-4e0b-a05d-fa03b2312272 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:27.055958 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1388/2759] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:26 2022] DELETE /identity/v3/projects/40e08dea949e4a27a3ebad1fab9359e3 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:27.059829 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f9590-5798-42f2-9e4a-07f6067e5ef8 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:27.071800 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f9590-5798-42f2-9e4a-07f6067e5ef8 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:27.131710 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fb195-5eeb-43db-8e52-20457578ee06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:27.132504 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fb195-5eeb-43db-8e52-20457578ee06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:27.132792 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fb195-5eeb-43db-8e52-20457578ee06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:27.133067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fb195-5eeb-43db-8e52-20457578ee06 admin admin] PATH_INFO: `/v3/users/9395c4c3576343fa8e55e16f02b42829` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:27.134429 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0fb195-5eeb-43db-8e52-20457578ee06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9395c4c3576343fa8e55e16f02b42829)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:27.137355 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0fb195-5eeb-43db-8e52-20457578ee06 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:27.381232 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9b0fb195-5eeb-43db-8e52-20457578ee06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9395c4c3576343fa8e55e16f02b42829'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:27.383332 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:27.388395 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9b0fb195-5eeb-43db-8e52-20457578ee06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9395c4c3576343fa8e55e16f02b42829'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:27.389074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9b0fb195-5eeb-43db-8e52-20457578ee06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9395c4c3576343fa8e55e16f02b42829'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:27.389679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9b0fb195-5eeb-43db-8e52-20457578ee06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9395c4c3576343fa8e55e16f02b42829'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:27.401909 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9b0fb195-5eeb-43db-8e52-20457578ee06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9395c4c3576343fa8e55e16f02b42829'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:27.556387 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1372/2760] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:27 2022] DELETE /identity/v3/users/9395c4c3576343fa8e55e16f02b42829 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:28.051264 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd83bc8-93dc-4e0b-a05d-fa03b2312272 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:28.053184 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-3bd83bc8-93dc-4e0b-a05d-fa03b2312272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:28.079814 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd83bc8-93dc-4e0b-a05d-fa03b2312272 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:28.100852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:28.141347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7611eea-ea61-4b07-a2a0-41008cb94694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:28.142208 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7611eea-ea61-4b07-a2a0-41008cb94694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:28.142488 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7611eea-ea61-4b07-a2a0-41008cb94694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:28.142788 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7611eea-ea61-4b07-a2a0-41008cb94694 admin admin] PATH_INFO: `/v3/projects/4d1432435f1d4af08367ca57dcd6e484` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:28.144146 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7611eea-ea61-4b07-a2a0-41008cb94694 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d1432435f1d4af08367ca57dcd6e484)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:28.146028 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7611eea-ea61-4b07-a2a0-41008cb94694 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:28.261963 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-b7611eea-ea61-4b07-a2a0-41008cb94694 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d1432435f1d4af08367ca57dcd6e484', 'request_id': 'req-b7611eea-ea61-4b07-a2a0-41008cb94694'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:28.268311 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-b7611eea-ea61-4b07-a2a0-41008cb94694 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d1432435f1d4af08367ca57dcd6e484', 'request_id': 'req-b7611eea-ea61-4b07-a2a0-41008cb94694'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:28.345121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-b7611eea-ea61-4b07-a2a0-41008cb94694 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d1432435f1d4af08367ca57dcd6e484', 'request_id': 'req-b7611eea-ea61-4b07-a2a0-41008cb94694'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:28.358190 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-b7611eea-ea61-4b07-a2a0-41008cb94694 admin admin] The token cache is being invalidate because project 4d1432435f1d4af08367ca57dcd6e484 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:28.358479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-b7611eea-ea61-4b07-a2a0-41008cb94694 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:28.359253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-b7611eea-ea61-4b07-a2a0-41008cb94694 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:28.360807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1389/2761] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:28 2022] DELETE /identity/v3/projects/4d1432435f1d4af08367ca57dcd6e484 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:28.365583 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fb195-5eeb-43db-8e52-20457578ee06 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:28.367108 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9b0fb195-5eeb-43db-8e52-20457578ee06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:28.380361 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fb195-5eeb-43db-8e52-20457578ee06 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:28.439896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8135f8b-f4db-4e33-8542-a292326a46a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:28.441526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8135f8b-f4db-4e33-8542-a292326a46a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:28.441808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8135f8b-f4db-4e33-8542-a292326a46a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:28.442082 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8135f8b-f4db-4e33-8542-a292326a46a0 admin admin] PATH_INFO: `/v3/users/10bc83eb7a46448ebc5740638e134250` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:28.443693 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8135f8b-f4db-4e33-8542-a292326a46a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10bc83eb7a46448ebc5740638e134250)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:28.445752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8135f8b-f4db-4e33-8542-a292326a46a0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:28.667782 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a8135f8b-f4db-4e33-8542-a292326a46a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10bc83eb7a46448ebc5740638e134250'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:28.674433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a8135f8b-f4db-4e33-8542-a292326a46a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10bc83eb7a46448ebc5740638e134250'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:28.675137 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a8135f8b-f4db-4e33-8542-a292326a46a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10bc83eb7a46448ebc5740638e134250'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:28.675756 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a8135f8b-f4db-4e33-8542-a292326a46a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10bc83eb7a46448ebc5740638e134250'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:28.687183 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a8135f8b-f4db-4e33-8542-a292326a46a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10bc83eb7a46448ebc5740638e134250'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:28.762018 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1373/2762] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:28 2022] DELETE /identity/v3/users/10bc83eb7a46448ebc5740638e134250 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:28.873241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b095a86-beed-4ea6-9d6e-a50be561cdc9 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:28.873533 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b095a86-beed-4ea6-9d6e-a50be561cdc9 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:28.873812 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b095a86-beed-4ea6-9d6e-a50be561cdc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:28.893220 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-5b095a86-beed-4ea6-9d6e-a50be561cdc9 None None] MFA Rules not processed for user `69c345b278e4437d98f9b1f412bbca49`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:28.895590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-5b095a86-beed-4ea6-9d6e-a50be561cdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:28.950353 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1390/2763] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:29.205217 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8135f8b-f4db-4e33-8542-a292326a46a0 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:29.206939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a8135f8b-f4db-4e33-8542-a292326a46a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:29.237128 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b095a86-beed-4ea6-9d6e-a50be561cdc9 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:29.239371 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-5b095a86-beed-4ea6-9d6e-a50be561cdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:29.268896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:29.274513 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8135f8b-f4db-4e33-8542-a292326a46a0 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:29.278367 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50a459-8481-4681-a656-b743fb35c982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:29.279131 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50a459-8481-4681-a656-b743fb35c982 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:29.279490 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50a459-8481-4681-a656-b743fb35c982 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:29.279847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50a459-8481-4681-a656-b743fb35c982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:29.281668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ba50a459-8481-4681-a656-b743fb35c982 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:29.291625 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50a459-8481-4681-a656-b743fb35c982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:29.293135 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50a459-8481-4681-a656-b743fb35c982 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:29.302242 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1374/2764] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:29.309720 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:29.315285 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b095a86-beed-4ea6-9d6e-a50be561cdc9 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:29.320926 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc8cf5-204c-4c08-9d5d-b5fb2fa31d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:29.321782 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc8cf5-204c-4c08-9d5d-b5fb2fa31d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:29.322062 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc8cf5-204c-4c08-9d5d-b5fb2fa31d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:29.322329 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc8cf5-204c-4c08-9d5d-b5fb2fa31d6e admin admin] PATH_INFO: `/v3/projects/b8abd1c4206e43bba04275f6f14ef7c5` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:29.323613 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc8cf5-204c-4c08-9d5d-b5fb2fa31d6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8abd1c4206e43bba04275f6f14ef7c5)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:29.325437 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc8cf5-204c-4c08-9d5d-b5fb2fa31d6e admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:29.435695 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d9cc8cf5-204c-4c08-9d5d-b5fb2fa31d6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8abd1c4206e43bba04275f6f14ef7c5', 'request_id': 'req-d9cc8cf5-204c-4c08-9d5d-b5fb2fa31d6e'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:29.443727 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d9cc8cf5-204c-4c08-9d5d-b5fb2fa31d6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8abd1c4206e43bba04275f6f14ef7c5', 'request_id': 'req-d9cc8cf5-204c-4c08-9d5d-b5fb2fa31d6e'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:29.544979 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d9cc8cf5-204c-4c08-9d5d-b5fb2fa31d6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8abd1c4206e43bba04275f6f14ef7c5', 'request_id': 'req-d9cc8cf5-204c-4c08-9d5d-b5fb2fa31d6e'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:29.556756 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d9cc8cf5-204c-4c08-9d5d-b5fb2fa31d6e admin admin] The token cache is being invalidate because project b8abd1c4206e43bba04275f6f14ef7c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:29.557043 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d9cc8cf5-204c-4c08-9d5d-b5fb2fa31d6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:29.557671 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-d9cc8cf5-204c-4c08-9d5d-b5fb2fa31d6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:29.559237 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1391/2765] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:29 2022] DELETE /identity/v3/projects/b8abd1c4206e43bba04275f6f14ef7c5 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:29.566777 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9957d-27d3-414d-b079-6271266d2286 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:29.567288 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9957d-27d3-414d-b079-6271266d2286 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:29.567408 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9957d-27d3-414d-b079-6271266d2286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:29.585139 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-27b9957d-27d3-414d-b079-6271266d2286 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:29.586341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-27b9957d-27d3-414d-b079-6271266d2286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:29.660382 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1375/2766] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:29.662063 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b269188-b29f-4df9-931b-2bfe54a1d969 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:29.662595 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b269188-b29f-4df9-931b-2bfe54a1d969 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:29.665546 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b269188-b29f-4df9-931b-2bfe54a1d969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:29.671672 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9957d-27d3-414d-b079-6271266d2286 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:29.673176 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-27b9957d-27d3-414d-b079-6271266d2286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:29.686157 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9957d-27d3-414d-b079-6271266d2286 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:29.689662 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-7b269188-b29f-4df9-931b-2bfe54a1d969 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:29.691042 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-7b269188-b29f-4df9-931b-2bfe54a1d969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:29.691271 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ad98c-4477-4dc5-b0d7-1915eb3c3e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:29.692143 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ad98c-4477-4dc5-b0d7-1915eb3c3e10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:29.692424 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ad98c-4477-4dc5-b0d7-1915eb3c3e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:29.693120 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ad98c-4477-4dc5-b0d7-1915eb3c3e10 admin admin] PATH_INFO: `/v3/users/9d3d9f50a04f4cf9aa7efebde17bc3ac` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:29.694326 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ad98c-4477-4dc5-b0d7-1915eb3c3e10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d3d9f50a04f4cf9aa7efebde17bc3ac)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:29.696513 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ad98c-4477-4dc5-b0d7-1915eb3c3e10 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:29.701055 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1392/2767] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:29.711957 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b269188-b29f-4df9-931b-2bfe54a1d969 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:29.721410 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b269188-b29f-4df9-931b-2bfe54a1d969 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:29.729369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb37391-e77e-4ecc-8f30-8e504058a396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:29.730331 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb37391-e77e-4ecc-8f30-8e504058a396 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:29.730917 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb37391-e77e-4ecc-8f30-8e504058a396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:29.731165 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb37391-e77e-4ecc-8f30-8e504058a396 admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:29.732231 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb37391-e77e-4ecc-8f30-8e504058a396 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:57:29.732708 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb37391-e77e-4ecc-8f30-8e504058a396 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:29.734453 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb37391-e77e-4ecc-8f30-8e504058a396 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:29.752800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1393/2768] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:57:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:57:29.754684 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb37391-e77e-4ecc-8f30-8e504058a396 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:29.764620 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb37391-e77e-4ecc-8f30-8e504058a396 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:29.769946 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beec792-8a27-47f6-a5c2-074c16591d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:29.770690 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beec792-8a27-47f6-a5c2-074c16591d21 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:29.770987 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beec792-8a27-47f6-a5c2-074c16591d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:29.771269 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beec792-8a27-47f6-a5c2-074c16591d21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:29.772039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beec792-8a27-47f6-a5c2-074c16591d21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:29.774097 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beec792-8a27-47f6-a5c2-074c16591d21 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:29.861018 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1394/2769] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:29 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:29.865936 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beec792-8a27-47f6-a5c2-074c16591d21 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:29.896169 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2c7ad98c-4477-4dc5-b0d7-1915eb3c3e10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d3d9f50a04f4cf9aa7efebde17bc3ac'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:29.903954 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2c7ad98c-4477-4dc5-b0d7-1915eb3c3e10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d3d9f50a04f4cf9aa7efebde17bc3ac'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:29.904738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2c7ad98c-4477-4dc5-b0d7-1915eb3c3e10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d3d9f50a04f4cf9aa7efebde17bc3ac'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:29.905354 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2c7ad98c-4477-4dc5-b0d7-1915eb3c3e10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d3d9f50a04f4cf9aa7efebde17bc3ac'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:29.919218 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2c7ad98c-4477-4dc5-b0d7-1915eb3c3e10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d3d9f50a04f4cf9aa7efebde17bc3ac'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:29.929644 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beec792-8a27-47f6-a5c2-074c16591d21 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:29.934427 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b0bdb-b200-4d72-9686-cb48e4bd1e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:29.935122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b0bdb-b200-4d72-9686-cb48e4bd1e32 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:29.935404 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b0bdb-b200-4d72-9686-cb48e4bd1e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:29.935680 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b0bdb-b200-4d72-9686-cb48e4bd1e32 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:29.936429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b0bdb-b200-4d72-9686-cb48e4bd1e32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:29.938431 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b0bdb-b200-4d72-9686-cb48e4bd1e32 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:30.020360 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1376/2770] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:29 2022] DELETE /identity/v3/users/9d3d9f50a04f4cf9aa7efebde17bc3ac => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:30.024637 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b1368-ed0d-47c4-b79e-493e9cb7bac6 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:30.024947 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b1368-ed0d-47c4-b79e-493e9cb7bac6 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:30.025225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b1368-ed0d-47c4-b79e-493e9cb7bac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:30.040840 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1395/2771] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:29 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:30.042130 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b0bdb-b200-4d72-9686-cb48e4bd1e32 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:30.043364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-055b1368-ed0d-47c4-b79e-493e9cb7bac6 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:30.043752 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-ab0b0bdb-b200-4d72-9686-cb48e4bd1e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:30.044553 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-055b1368-ed0d-47c4-b79e-493e9cb7bac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:30.055755 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b0bdb-b200-4d72-9686-cb48e4bd1e32 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:30.087048 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:30.106628 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1377/2772] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:30.114803 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20005c01-7532-478b-aab0-9c3125036622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:30.115424 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20005c01-7532-478b-aab0-9c3125036622 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:30.115996 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20005c01-7532-478b-aab0-9c3125036622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:30.116405 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20005c01-7532-478b-aab0-9c3125036622 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:30.117342 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20005c01-7532-478b-aab0-9c3125036622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:30.117471 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b1368-ed0d-47c4-b79e-493e9cb7bac6 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:30.118774 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-055b1368-ed0d-47c4-b79e-493e9cb7bac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:30.119435 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20005c01-7532-478b-aab0-9c3125036622 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:30.124056 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:30.132347 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1396/2773] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:30.133523 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20005c01-7532-478b-aab0-9c3125036622 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:30.143003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20005c01-7532-478b-aab0-9c3125036622 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:30.148280 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f974e6-f864-4868-8a34-b8289125f659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:30.148918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f974e6-f864-4868-8a34-b8289125f659 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:30.149212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f974e6-f864-4868-8a34-b8289125f659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:30.149485 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f974e6-f864-4868-8a34-b8289125f659 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:30.150206 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f974e6-f864-4868-8a34-b8289125f659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:30.152284 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f974e6-f864-4868-8a34-b8289125f659 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:30.162956 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1397/2774] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:30.167194 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f974e6-f864-4868-8a34-b8289125f659 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:30.176885 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f974e6-f864-4868-8a34-b8289125f659 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:30.177074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b1368-ed0d-47c4-b79e-493e9cb7bac6 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:30.180646 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364ef6b-51c9-4e33-b9d4-1f42c24d58a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:30.181289 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364ef6b-51c9-4e33-b9d4-1f42c24d58a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:30.181995 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364ef6b-51c9-4e33-b9d4-1f42c24d58a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:30.182275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364ef6b-51c9-4e33-b9d4-1f42c24d58a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:30.183714 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30edb8-c882-40d9-b49f-604d61e9d3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:30.184581 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30edb8-c882-40d9-b49f-604d61e9d3ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:30.184864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30edb8-c882-40d9-b49f-604d61e9d3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:30.185140 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30edb8-c882-40d9-b49f-604d61e9d3ca admin admin] PATH_INFO: `/v3/projects/ef7e839a304c42d79b01004f5518d1f6/users/c740f460f4ad40d8a1a78efadb239adf/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:30.192390 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364ef6b-51c9-4e33-b9d4-1f42c24d58a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:30.193886 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364ef6b-51c9-4e33-b9d4-1f42c24d58a2 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:30.199888 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30edb8-c882-40d9-b49f-604d61e9d3ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef7e839a304c42d79b01004f5518d1f6, user_id=c740f460f4ad40d8a1a78efadb239adf, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:30.202514 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30edb8-c882-40d9-b49f-604d61e9d3ca admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:30.203609 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1378/2775] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:57:30.317545 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1398/2776] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:30 2022] PUT /identity/v3/projects/ef7e839a304c42d79b01004f5518d1f6/users/c740f460f4ad40d8a1a78efadb239adf/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:30.328797 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96182557-1df7-498e-ac69-5fc8cd9ff963 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:30.329109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96182557-1df7-498e-ac69-5fc8cd9ff963 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:30.329384 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96182557-1df7-498e-ac69-5fc8cd9ff963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:30.357105 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-96182557-1df7-498e-ac69-5fc8cd9ff963 None None] MFA Rules not processed for user `c740f460f4ad40d8a1a78efadb239adf`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:30.358307 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-96182557-1df7-498e-ac69-5fc8cd9ff963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:30.422443 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1379/2777] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:30.427211 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30edb8-c882-40d9-b49f-604d61e9d3ca admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:30.498793 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30edb8-c882-40d9-b49f-604d61e9d3ca admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:30.504151 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fbd9ea-64a6-4944-9486-f39ee7f04b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:30.504924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbd9ea-64a6-4944-9486-f39ee7f04b2e admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:30.505202 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbd9ea-64a6-4944-9486-f39ee7f04b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:30.505482 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbd9ea-64a6-4944-9486-f39ee7f04b2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:30.506189 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fbd9ea-64a6-4944-9486-f39ee7f04b2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:30.508157 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fbd9ea-64a6-4944-9486-f39ee7f04b2e admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:30.648844 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1399/2778] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:30 2022] POST /identity/v3/projects => generated 383 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:30.652991 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96182557-1df7-498e-ac69-5fc8cd9ff963 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:30.663677 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96182557-1df7-498e-ac69-5fc8cd9ff963 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:30.672359 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fbd9ea-64a6-4944-9486-f39ee7f04b2e admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:30.677052 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d7fbd9ea-64a6-4944-9486-f39ee7f04b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:30.747157 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a060e68-a624-4ff7-adb2-e44d38629e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:30.747852 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a060e68-a624-4ff7-adb2-e44d38629e8e admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:30.748132 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a060e68-a624-4ff7-adb2-e44d38629e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:30.748409 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a060e68-a624-4ff7-adb2-e44d38629e8e admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:30.749102 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a060e68-a624-4ff7-adb2-e44d38629e8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:30.749569 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fbd9ea-64a6-4944-9486-f39ee7f04b2e admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:30.751036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a060e68-a624-4ff7-adb2-e44d38629e8e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:30.755343 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738156a-b459-43c9-8927-8ca5c5ca46c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:30.756286 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738156a-b459-43c9-8927-8ca5c5ca46c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:30.756639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738156a-b459-43c9-8927-8ca5c5ca46c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:30.756986 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738156a-b459-43c9-8927-8ca5c5ca46c4 admin admin] PATH_INFO: `/v3/projects/51d6d52f57ea42b299be69ae9f80e78e` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:30.758247 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738156a-b459-43c9-8927-8ca5c5ca46c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51d6d52f57ea42b299be69ae9f80e78e)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:30.760334 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738156a-b459-43c9-8927-8ca5c5ca46c4 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:30.790563 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:30.842740 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1380/2779] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:30 2022] POST /identity/v3/users => generated 298 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:30.846511 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a060e68-a624-4ff7-adb2-e44d38629e8e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:30.862392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a060e68-a624-4ff7-adb2-e44d38629e8e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:30.868200 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fda03-9e77-4d14-a3ba-169cc0434747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:30.868857 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fda03-9e77-4d14-a3ba-169cc0434747 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:30.869414 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fda03-9e77-4d14-a3ba-169cc0434747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:30.869690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fda03-9e77-4d14-a3ba-169cc0434747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:30.870417 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fda03-9e77-4d14-a3ba-169cc0434747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:30.872033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fda03-9e77-4d14-a3ba-169cc0434747 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:30.883524 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1381/2780] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:30.887420 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fda03-9e77-4d14-a3ba-169cc0434747 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:30.889410 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3738156a-b459-43c9-8927-8ca5c5ca46c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51d6d52f57ea42b299be69ae9f80e78e', 'request_id': 'req-3738156a-b459-43c9-8927-8ca5c5ca46c4'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:30.895619 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3738156a-b459-43c9-8927-8ca5c5ca46c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51d6d52f57ea42b299be69ae9f80e78e', 'request_id': 'req-3738156a-b459-43c9-8927-8ca5c5ca46c4'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:30.897374 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fda03-9e77-4d14-a3ba-169cc0434747 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:30.906172 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:30.954598 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f41c20-796f-4e35-898e-4e59532b77bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:30.955488 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f41c20-796f-4e35-898e-4e59532b77bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:30.955810 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f41c20-796f-4e35-898e-4e59532b77bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:30.956038 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f41c20-796f-4e35-898e-4e59532b77bc admin admin] PATH_INFO: `/v3/projects/e0b5350aba5d4eaa8af5a2da06e2ecd4/users/af27a05376364aa9993755faa904bf08/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:30.970486 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f41c20-796f-4e35-898e-4e59532b77bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0b5350aba5d4eaa8af5a2da06e2ecd4, user_id=af27a05376364aa9993755faa904bf08, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:30.973209 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f41c20-796f-4e35-898e-4e59532b77bc admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:31.007468 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3738156a-b459-43c9-8927-8ca5c5ca46c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51d6d52f57ea42b299be69ae9f80e78e', 'request_id': 'req-3738156a-b459-43c9-8927-8ca5c5ca46c4'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:31.018726 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3738156a-b459-43c9-8927-8ca5c5ca46c4 admin admin] The token cache is being invalidate because project 51d6d52f57ea42b299be69ae9f80e78e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:31.019322 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3738156a-b459-43c9-8927-8ca5c5ca46c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:31.019933 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-3738156a-b459-43c9-8927-8ca5c5ca46c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:31.021498 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1400/2781] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:30 2022] DELETE /identity/v3/projects/51d6d52f57ea42b299be69ae9f80e78e => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:31.025256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738156a-b459-43c9-8927-8ca5c5ca46c4 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:31.026640 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-3738156a-b459-43c9-8927-8ca5c5ca46c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:31.039698 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738156a-b459-43c9-8927-8ca5c5ca46c4 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:31.045562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb781dd4-c8e7-4f92-82d5-cbca3b0477fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:31.045562 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb781dd4-c8e7-4f92-82d5-cbca3b0477fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:31.045801 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb781dd4-c8e7-4f92-82d5-cbca3b0477fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:31.046034 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb781dd4-c8e7-4f92-82d5-cbca3b0477fc admin admin] PATH_INFO: `/v3/users/c46b4678cabe42d5b504745d4feb7f14` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:31.047444 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb781dd4-c8e7-4f92-82d5-cbca3b0477fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c46b4678cabe42d5b504745d4feb7f14)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:31.049466 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb781dd4-c8e7-4f92-82d5-cbca3b0477fc admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:31.054612 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1382/2782] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:30 2022] PUT /identity/v3/projects/e0b5350aba5d4eaa8af5a2da06e2ecd4/users/af27a05376364aa9993755faa904bf08/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:31.058423 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f41c20-796f-4e35-898e-4e59532b77bc admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:31.060080 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d4f41c20-796f-4e35-898e-4e59532b77bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:31.124184 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f41c20-796f-4e35-898e-4e59532b77bc admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:31.128884 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8acd4ca-daa0-4ca5-a8a2-cd517c0b5d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:31.129557 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acd4ca-daa0-4ca5-a8a2-cd517c0b5d50 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:31.129839 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acd4ca-daa0-4ca5-a8a2-cd517c0b5d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:31.130115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acd4ca-daa0-4ca5-a8a2-cd517c0b5d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:31.130873 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8acd4ca-daa0-4ca5-a8a2-cd517c0b5d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:31.132469 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8acd4ca-daa0-4ca5-a8a2-cd517c0b5d50 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:31.142171 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1383/2783] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:31.146130 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8acd4ca-daa0-4ca5-a8a2-cd517c0b5d50 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:31.155392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8acd4ca-daa0-4ca5-a8a2-cd517c0b5d50 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:31.160633 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23687ee-a853-4d80-b1f1-2a36dbb05741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:31.161450 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23687ee-a853-4d80-b1f1-2a36dbb05741 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:31.161735 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23687ee-a853-4d80-b1f1-2a36dbb05741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:31.162021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23687ee-a853-4d80-b1f1-2a36dbb05741 admin admin] PATH_INFO: `/v3/projects/e0b5350aba5d4eaa8af5a2da06e2ecd4/users/af27a05376364aa9993755faa904bf08/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:31.164013 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23687ee-a853-4d80-b1f1-2a36dbb05741 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0b5350aba5d4eaa8af5a2da06e2ecd4, user_id=af27a05376364aa9993755faa904bf08, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:31.166663 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23687ee-a853-4d80-b1f1-2a36dbb05741 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:31.177359 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1384/2784] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:31 2022] PUT /identity/v3/projects/e0b5350aba5d4eaa8af5a2da06e2ecd4/users/af27a05376364aa9993755faa904bf08/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:31.187149 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0e737-b49a-417d-9db4-132dbf42aafd None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:31.187442 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0e737-b49a-417d-9db4-132dbf42aafd None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:31.187723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0e737-b49a-417d-9db4-132dbf42aafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:31.213522 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-24a0e737-b49a-417d-9db4-132dbf42aafd None None] MFA Rules not processed for user `af27a05376364aa9993755faa904bf08`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:31.214745 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-24a0e737-b49a-417d-9db4-132dbf42aafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:31.250253 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-bb781dd4-c8e7-4f92-82d5-cbca3b0477fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c46b4678cabe42d5b504745d4feb7f14'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:31.257868 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-bb781dd4-c8e7-4f92-82d5-cbca3b0477fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c46b4678cabe42d5b504745d4feb7f14'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:31.258579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-bb781dd4-c8e7-4f92-82d5-cbca3b0477fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c46b4678cabe42d5b504745d4feb7f14'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:31.259203 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-bb781dd4-c8e7-4f92-82d5-cbca3b0477fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c46b4678cabe42d5b504745d4feb7f14'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:31.271526 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-bb781dd4-c8e7-4f92-82d5-cbca3b0477fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c46b4678cabe42d5b504745d4feb7f14'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:31.285106 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1385/2785] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:31.296341 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0e737-b49a-417d-9db4-132dbf42aafd None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:31.297716 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-24a0e737-b49a-417d-9db4-132dbf42aafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:31.304633 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:31.353407 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0e737-b49a-417d-9db4-132dbf42aafd None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:31.356212 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1401/2786] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:31 2022] DELETE /identity/v3/users/c46b4678cabe42d5b504745d4feb7f14 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:31.357706 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdad421-ae0f-4666-a4d3-38e1cb93d99e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:31.358380 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdad421-ae0f-4666-a4d3-38e1cb93d99e service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:31.358698 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdad421-ae0f-4666-a4d3-38e1cb93d99e service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:31.358971 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdad421-ae0f-4666-a4d3-38e1cb93d99e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:31.369710 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdad421-ae0f-4666-a4d3-38e1cb93d99e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:31.371191 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdad421-ae0f-4666-a4d3-38e1cb93d99e service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:31.382694 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1386/2787] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:57:31.800987 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb781dd4-c8e7-4f92-82d5-cbca3b0477fc admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:31.800987 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-bb781dd4-c8e7-4f92-82d5-cbca3b0477fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:31.921594 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb781dd4-c8e7-4f92-82d5-cbca3b0477fc admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:31.926582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c41397-d97d-4c52-89a2-90d2720dd9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:31.932302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c41397-d97d-4c52-89a2-90d2720dd9df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:31.932302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c41397-d97d-4c52-89a2-90d2720dd9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:31.932302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c41397-d97d-4c52-89a2-90d2720dd9df admin admin] PATH_INFO: `/v3/projects/2f2144028cfc48a28f0799edfe60354a` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:31.932302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c41397-d97d-4c52-89a2-90d2720dd9df admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f2144028cfc48a28f0799edfe60354a)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:31.932302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c41397-d97d-4c52-89a2-90d2720dd9df admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:31.958446 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:32.074197 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c9c41397-d97d-4c52-89a2-90d2720dd9df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f2144028cfc48a28f0799edfe60354a', 'request_id': 'req-c9c41397-d97d-4c52-89a2-90d2720dd9df'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:32.080959 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c9c41397-d97d-4c52-89a2-90d2720dd9df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f2144028cfc48a28f0799edfe60354a', 'request_id': 'req-c9c41397-d97d-4c52-89a2-90d2720dd9df'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:32.157834 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c9c41397-d97d-4c52-89a2-90d2720dd9df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f2144028cfc48a28f0799edfe60354a', 'request_id': 'req-c9c41397-d97d-4c52-89a2-90d2720dd9df'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:32.169311 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c9c41397-d97d-4c52-89a2-90d2720dd9df admin admin] The token cache is being invalidate because project 2f2144028cfc48a28f0799edfe60354a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:32.169616 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c9c41397-d97d-4c52-89a2-90d2720dd9df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:32.170251 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-c9c41397-d97d-4c52-89a2-90d2720dd9df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:32.171929 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1402/2788] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:31 2022] DELETE /identity/v3/projects/2f2144028cfc48a28f0799edfe60354a => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:32.176609 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdad421-ae0f-4666-a4d3-38e1cb93d99e service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:32.255663 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdad421-ae0f-4666-a4d3-38e1cb93d99e service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:32.315770 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a991c-7d45-404d-9ecd-a4b2e8fc9d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:32.317246 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a991c-7d45-404d-9ecd-a4b2e8fc9d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:32.317534 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a991c-7d45-404d-9ecd-a4b2e8fc9d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:32.317815 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a991c-7d45-404d-9ecd-a4b2e8fc9d1f admin admin] PATH_INFO: `/v3/users/fb0de0396503407490b97acdc7c1faad` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:32.319107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a991c-7d45-404d-9ecd-a4b2e8fc9d1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb0de0396503407490b97acdc7c1faad)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:32.321022 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a991c-7d45-404d-9ecd-a4b2e8fc9d1f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:32.507630 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f23a991c-7d45-404d-9ecd-a4b2e8fc9d1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb0de0396503407490b97acdc7c1faad'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:32.514925 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f23a991c-7d45-404d-9ecd-a4b2e8fc9d1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb0de0396503407490b97acdc7c1faad'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:32.515597 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f23a991c-7d45-404d-9ecd-a4b2e8fc9d1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb0de0396503407490b97acdc7c1faad'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:32.516222 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f23a991c-7d45-404d-9ecd-a4b2e8fc9d1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb0de0396503407490b97acdc7c1faad'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:32.517370 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:32.528714 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f23a991c-7d45-404d-9ecd-a4b2e8fc9d1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb0de0396503407490b97acdc7c1faad'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:32.631830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1387/2789] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:32 2022] DELETE /identity/v3/users/fb0de0396503407490b97acdc7c1faad => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:32.633968 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c41397-d97d-4c52-89a2-90d2720dd9df admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:32.635987 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c9c41397-d97d-4c52-89a2-90d2720dd9df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:32.648771 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c41397-d97d-4c52-89a2-90d2720dd9df admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:32.706633 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d227e-3534-4fa8-aae3-e53ee51db933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:32.707497 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d227e-3534-4fa8-aae3-e53ee51db933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:32.707779 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d227e-3534-4fa8-aae3-e53ee51db933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:32.708085 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d227e-3534-4fa8-aae3-e53ee51db933 admin admin] PATH_INFO: `/v3/users/c740f460f4ad40d8a1a78efadb239adf` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:32.709429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d227e-3534-4fa8-aae3-e53ee51db933 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c740f460f4ad40d8a1a78efadb239adf)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:32.711473 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d227e-3534-4fa8-aae3-e53ee51db933 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:33.000540 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7a3d227e-3534-4fa8-aae3-e53ee51db933 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c740f460f4ad40d8a1a78efadb239adf'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:33.012660 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7a3d227e-3534-4fa8-aae3-e53ee51db933 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c740f460f4ad40d8a1a78efadb239adf'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:33.013719 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7a3d227e-3534-4fa8-aae3-e53ee51db933 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c740f460f4ad40d8a1a78efadb239adf'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:33.014704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7a3d227e-3534-4fa8-aae3-e53ee51db933 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c740f460f4ad40d8a1a78efadb239adf'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:33.031142 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7a3d227e-3534-4fa8-aae3-e53ee51db933 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c740f460f4ad40d8a1a78efadb239adf'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:33.113235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a991c-7d45-404d-9ecd-a4b2e8fc9d1f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:33.114760 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f23a991c-7d45-404d-9ecd-a4b2e8fc9d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:33.130061 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a991c-7d45-404d-9ecd-a4b2e8fc9d1f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:33.137071 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1403/2790] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:32 2022] DELETE /identity/v3/users/c740f460f4ad40d8a1a78efadb239adf => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:33.140529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2299c0-d876-4f65-a427-0e0aa95d2715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:33.141520 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2299c0-d876-4f65-a427-0e0aa95d2715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:33.141915 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2299c0-d876-4f65-a427-0e0aa95d2715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:33.142278 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2299c0-d876-4f65-a427-0e0aa95d2715 admin admin] PATH_INFO: `/v3/projects/a89f70db6f184174842f7b50b62c6e42` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:33.142837 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ba9b3-a2ac-4880-85cb-af51a1047932 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:33.146621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2299c0-d876-4f65-a427-0e0aa95d2715 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a89f70db6f184174842f7b50b62c6e42)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:33.146621 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2299c0-d876-4f65-a427-0e0aa95d2715 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:33.149254 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ba9b3-a2ac-4880-85cb-af51a1047932 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:33.150271 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ba9b3-a2ac-4880-85cb-af51a1047932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:33.174503 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c20ba9b3-a2ac-4880-85cb-af51a1047932 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:33.176123 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c20ba9b3-a2ac-4880-85cb-af51a1047932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:33.203650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:33.239323 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1404/2791] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:33.248938 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ba9b3-a2ac-4880-85cb-af51a1047932 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:33.250324 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c20ba9b3-a2ac-4880-85cb-af51a1047932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:33.258306 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2b2299c0-d876-4f65-a427-0e0aa95d2715 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a89f70db6f184174842f7b50b62c6e42', 'request_id': 'req-2b2299c0-d876-4f65-a427-0e0aa95d2715'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:33.264597 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2b2299c0-d876-4f65-a427-0e0aa95d2715 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a89f70db6f184174842f7b50b62c6e42', 'request_id': 'req-2b2299c0-d876-4f65-a427-0e0aa95d2715'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:33.307828 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ba9b3-a2ac-4880-85cb-af51a1047932 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:33.349198 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2b2299c0-d876-4f65-a427-0e0aa95d2715 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a89f70db6f184174842f7b50b62c6e42', 'request_id': 'req-2b2299c0-d876-4f65-a427-0e0aa95d2715'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:33.359813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd928f-63d5-4ef2-84f7-7f6f46c5a68b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:33.360593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd928f-63d5-4ef2-84f7-7f6f46c5a68b service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:33.360877 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd928f-63d5-4ef2-84f7-7f6f46c5a68b service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:33.361152 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd928f-63d5-4ef2-84f7-7f6f46c5a68b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:33.362919 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2b2299c0-d876-4f65-a427-0e0aa95d2715 admin admin] The token cache is being invalidate because project a89f70db6f184174842f7b50b62c6e42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:33.363256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2b2299c0-d876-4f65-a427-0e0aa95d2715 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:33.364299 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2b2299c0-d876-4f65-a427-0e0aa95d2715 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:33.369088 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1388/2792] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:33 2022] DELETE /identity/v3/projects/a89f70db6f184174842f7b50b62c6e42 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:33.370938 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2526b-8ee6-4d59-9f35-7e1e88e94f92 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:33.371227 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2526b-8ee6-4d59-9f35-7e1e88e94f92 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:33.371804 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2526b-8ee6-4d59-9f35-7e1e88e94f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:33.391352 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-6fe2526b-8ee6-4d59-9f35-7e1e88e94f92 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:33.392501 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6fe2526b-8ee6-4d59-9f35-7e1e88e94f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:33.424961 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd928f-63d5-4ef2-84f7-7f6f46c5a68b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:33.427043 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd928f-63d5-4ef2-84f7-7f6f46c5a68b service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:33.437692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1405/2793] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:57:33.454571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1389/2794] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:33.519508 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64651f6-9776-4d25-9c97-8b4a8f605def None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:33.520308 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64651f6-9776-4d25-9c97-8b4a8f605def None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:33.520308 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64651f6-9776-4d25-9c97-8b4a8f605def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:33.539218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-d64651f6-9776-4d25-9c97-8b4a8f605def None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:33.540434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-d64651f6-9776-4d25-9c97-8b4a8f605def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:33.548747 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1406/2795] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:33.553519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe2526b-8ee6-4d59-9f35-7e1e88e94f92 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:33.563534 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe2526b-8ee6-4d59-9f35-7e1e88e94f92 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:33.568543 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c34732-6bd6-429b-8c18-b1913d29824d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:33.569198 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c34732-6bd6-429b-8c18-b1913d29824d admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:33.569483 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c34732-6bd6-429b-8c18-b1913d29824d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:33.569761 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c34732-6bd6-429b-8c18-b1913d29824d admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:33.570799 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c34732-6bd6-429b-8c18-b1913d29824d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:57:33.571111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c34732-6bd6-429b-8c18-b1913d29824d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:33.572702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c34732-6bd6-429b-8c18-b1913d29824d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:33.584431 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1390/2796] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:57:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:57:33.588752 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64651f6-9776-4d25-9c97-8b4a8f605def None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:33.598665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64651f6-9776-4d25-9c97-8b4a8f605def None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:33.603361 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92d758-2f63-4af2-9036-d9e2d25076c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:33.604160 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92d758-2f63-4af2-9036-d9e2d25076c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:33.604451 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92d758-2f63-4af2-9036-d9e2d25076c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:33.604732 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92d758-2f63-4af2-9036-d9e2d25076c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:33.605418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe92d758-2f63-4af2-9036-d9e2d25076c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:33.607645 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe92d758-2f63-4af2-9036-d9e2d25076c0 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:33.727394 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1407/2797] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:33 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:33.728736 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c34732-6bd6-429b-8c18-b1913d29824d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:33.740349 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c34732-6bd6-429b-8c18-b1913d29824d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:33.789635 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:33.804374 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff6280-0d6c-4c21-aec5-7567b207cde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:33.805041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff6280-0d6c-4c21-aec5-7567b207cde7 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:33.805324 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff6280-0d6c-4c21-aec5-7567b207cde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:33.805595 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff6280-0d6c-4c21-aec5-7567b207cde7 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:33.806334 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff6280-0d6c-4c21-aec5-7567b207cde7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:33.808312 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff6280-0d6c-4c21-aec5-7567b207cde7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:33.951379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1391/2798] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:33 2022] POST /identity/v3/users => generated 292 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:33.951500 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92d758-2f63-4af2-9036-d9e2d25076c0 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:33.951500 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-fe92d758-2f63-4af2-9036-d9e2d25076c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:33.951697 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff6280-0d6c-4c21-aec5-7567b207cde7 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:33.958182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff6280-0d6c-4c21-aec5-7567b207cde7 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:33.961041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92d758-2f63-4af2-9036-d9e2d25076c0 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:33.962874 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf464a-1ef6-4bd6-84a2-63b3ffab51da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:33.963539 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf464a-1ef6-4bd6-84a2-63b3ffab51da admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:33.963819 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf464a-1ef6-4bd6-84a2-63b3ffab51da admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:33.964090 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf464a-1ef6-4bd6-84a2-63b3ffab51da admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:33.964808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf464a-1ef6-4bd6-84a2-63b3ffab51da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:33.966392 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf464a-1ef6-4bd6-84a2-63b3ffab51da admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:33.967921 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e0d0c-271c-498d-b883-3e9df8b183fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:33.968867 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e0d0c-271c-498d-b883-3e9df8b183fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:33.969132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e0d0c-271c-498d-b883-3e9df8b183fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:33.969407 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e0d0c-271c-498d-b883-3e9df8b183fb admin admin] PATH_INFO: `/v3/projects/ef7e839a304c42d79b01004f5518d1f6` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:33.970842 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e0d0c-271c-498d-b883-3e9df8b183fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef7e839a304c42d79b01004f5518d1f6)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:33.973486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e0d0c-271c-498d-b883-3e9df8b183fb admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:33.977048 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:33.977699 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1392/2799] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:33.981257 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf464a-1ef6-4bd6-84a2-63b3ffab51da admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:33.991021 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf464a-1ef6-4bd6-84a2-63b3ffab51da admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:33.995757 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10c63f-2652-4f5c-9835-73f7d439463b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:33.996406 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10c63f-2652-4f5c-9835-73f7d439463b admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:33.996688 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10c63f-2652-4f5c-9835-73f7d439463b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:33.999347 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10c63f-2652-4f5c-9835-73f7d439463b admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:33.999347 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10c63f-2652-4f5c-9835-73f7d439463b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:33.999347 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10c63f-2652-4f5c-9835-73f7d439463b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:34.010836 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1393/2800] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:34.014987 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10c63f-2652-4f5c-9835-73f7d439463b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:34.025469 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10c63f-2652-4f5c-9835-73f7d439463b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:34.037151 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0fdec-6b91-47b1-a762-a99fc2724783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:34.037151 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0fdec-6b91-47b1-a762-a99fc2724783 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:34.037151 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0fdec-6b91-47b1-a762-a99fc2724783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:34.037151 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0fdec-6b91-47b1-a762-a99fc2724783 admin admin] PATH_INFO: `/v3/projects/d6e6f7e0d03a4e968775df1aa547cfbf/users/a673ea6aebc241fb9c9f5037c4d72b0e/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:34.045282 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0fdec-6b91-47b1-a762-a99fc2724783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6e6f7e0d03a4e968775df1aa547cfbf, user_id=a673ea6aebc241fb9c9f5037c4d72b0e, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:34.047953 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0fdec-6b91-47b1-a762-a99fc2724783 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:34.173996 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1394/2801] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:34 2022] PUT /identity/v3/projects/d6e6f7e0d03a4e968775df1aa547cfbf/users/a673ea6aebc241fb9c9f5037c4d72b0e/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:34.183146 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7e1e0d0c-271c-498d-b883-3e9df8b183fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef7e839a304c42d79b01004f5518d1f6', 'request_id': 'req-7e1e0d0c-271c-498d-b883-3e9df8b183fb'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:34.188884 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d028a-1ae3-4a2b-b9d0-7689bfd16e91 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:34.189116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d028a-1ae3-4a2b-b9d0-7689bfd16e91 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:34.189400 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d028a-1ae3-4a2b-b9d0-7689bfd16e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:34.189773 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7e1e0d0c-271c-498d-b883-3e9df8b183fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef7e839a304c42d79b01004f5518d1f6', 'request_id': 'req-7e1e0d0c-271c-498d-b883-3e9df8b183fb'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:34.216329 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a72d028a-1ae3-4a2b-b9d0-7689bfd16e91 None None] MFA Rules not processed for user `a673ea6aebc241fb9c9f5037c4d72b0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:34.217525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a72d028a-1ae3-4a2b-b9d0-7689bfd16e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:34.280182 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7e1e0d0c-271c-498d-b883-3e9df8b183fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef7e839a304c42d79b01004f5518d1f6', 'request_id': 'req-7e1e0d0c-271c-498d-b883-3e9df8b183fb'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:34.281519 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1395/2802] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:34.292140 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7e1e0d0c-271c-498d-b883-3e9df8b183fb admin admin] The token cache is being invalidate because project ef7e839a304c42d79b01004f5518d1f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:34.292429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7e1e0d0c-271c-498d-b883-3e9df8b183fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:34.292907 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d028a-1ae3-4a2b-b9d0-7689bfd16e91 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:34.293070 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-7e1e0d0c-271c-498d-b883-3e9df8b183fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:34.294454 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a72d028a-1ae3-4a2b-b9d0-7689bfd16e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:34.294692 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1408/2803] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:33 2022] DELETE /identity/v3/projects/ef7e839a304c42d79b01004f5518d1f6 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:34.298276 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e0d0c-271c-498d-b883-3e9df8b183fb admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:34.299548 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-7e1e0d0c-271c-498d-b883-3e9df8b183fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:34.353270 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d028a-1ae3-4a2b-b9d0-7689bfd16e91 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:34.357231 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5834fa9-0377-4ecd-9520-988dda393472 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:34.357898 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5834fa9-0377-4ecd-9520-988dda393472 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:34.358203 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5834fa9-0377-4ecd-9520-988dda393472 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:34.358556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5834fa9-0377-4ecd-9520-988dda393472 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:34.360399 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c5834fa9-0377-4ecd-9520-988dda393472 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:34.363089 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e0d0c-271c-498d-b883-3e9df8b183fb admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:34.368254 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ef6e1-241a-4b07-bfa7-80f28374854f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:34.369118 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ef6e1-241a-4b07-bfa7-80f28374854f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:34.369400 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ef6e1-241a-4b07-bfa7-80f28374854f admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:34.369676 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ef6e1-241a-4b07-bfa7-80f28374854f admin admin] PATH_INFO: `/v3/users/af27a05376364aa9993755faa904bf08` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:34.371036 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ef6e1-241a-4b07-bfa7-80f28374854f admin admin] RBAC: Authorizing `identity:delete_user(user_id=af27a05376364aa9993755faa904bf08)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:34.371609 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5834fa9-0377-4ecd-9520-988dda393472 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:34.372984 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ef6e1-241a-4b07-bfa7-80f28374854f admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:34.373128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5834fa9-0377-4ecd-9520-988dda393472 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:34.382734 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1396/2804] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:57:34.576127 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6e0ef6e1-241a-4b07-bfa7-80f28374854f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af27a05376364aa9993755faa904bf08'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:34.583130 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6e0ef6e1-241a-4b07-bfa7-80f28374854f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af27a05376364aa9993755faa904bf08'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:34.583805 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6e0ef6e1-241a-4b07-bfa7-80f28374854f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af27a05376364aa9993755faa904bf08'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:34.584427 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6e0ef6e1-241a-4b07-bfa7-80f28374854f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af27a05376364aa9993755faa904bf08'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:34.599839 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6e0ef6e1-241a-4b07-bfa7-80f28374854f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af27a05376364aa9993755faa904bf08'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:34.710433 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1409/2805] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:34 2022] DELETE /identity/v3/users/af27a05376364aa9993755faa904bf08 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:35.174565 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5834fa9-0377-4ecd-9520-988dda393472 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:35.190411 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5834fa9-0377-4ecd-9520-988dda393472 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:35.219198 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:35.265063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56ed30-4a28-4e27-88af-b40568201836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:35.265935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56ed30-4a28-4e27-88af-b40568201836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:35.266223 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56ed30-4a28-4e27-88af-b40568201836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:35.266502 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56ed30-4a28-4e27-88af-b40568201836 admin admin] PATH_INFO: `/v3/projects/e0b5350aba5d4eaa8af5a2da06e2ecd4` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:35.267720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56ed30-4a28-4e27-88af-b40568201836 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0b5350aba5d4eaa8af5a2da06e2ecd4)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:35.270523 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56ed30-4a28-4e27-88af-b40568201836 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:35.438110 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0d56ed30-4a28-4e27-88af-b40568201836 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0b5350aba5d4eaa8af5a2da06e2ecd4', 'request_id': 'req-0d56ed30-4a28-4e27-88af-b40568201836'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:35.444759 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0d56ed30-4a28-4e27-88af-b40568201836 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0b5350aba5d4eaa8af5a2da06e2ecd4', 'request_id': 'req-0d56ed30-4a28-4e27-88af-b40568201836'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:35.497693 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0d56ed30-4a28-4e27-88af-b40568201836 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0b5350aba5d4eaa8af5a2da06e2ecd4', 'request_id': 'req-0d56ed30-4a28-4e27-88af-b40568201836'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:35.510770 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0d56ed30-4a28-4e27-88af-b40568201836 admin admin] The token cache is being invalidate because project e0b5350aba5d4eaa8af5a2da06e2ecd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:35.511061 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0d56ed30-4a28-4e27-88af-b40568201836 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:35.511699 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0d56ed30-4a28-4e27-88af-b40568201836 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:35.513248 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1397/2806] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:35 2022] DELETE /identity/v3/projects/e0b5350aba5d4eaa8af5a2da06e2ecd4 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:35.518036 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49476d1-a492-416a-8af0-5bb83d3f1468 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:35.518519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49476d1-a492-416a-8af0-5bb83d3f1468 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:35.518636 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49476d1-a492-416a-8af0-5bb83d3f1468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:35.537863 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c49476d1-a492-416a-8af0-5bb83d3f1468 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:35.539192 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c49476d1-a492-416a-8af0-5bb83d3f1468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:35.544544 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:35.596916 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1410/2807] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:35.604505 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8103d-292c-43ec-9614-7f43d6c46f05 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:35.604791 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8103d-292c-43ec-9614-7f43d6c46f05 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:35.605068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8103d-292c-43ec-9614-7f43d6c46f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:35.623743 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-36e8103d-292c-43ec-9614-7f43d6c46f05 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:35.624922 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-36e8103d-292c-43ec-9614-7f43d6c46f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:35.633680 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1398/2808] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:35.638272 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49476d1-a492-416a-8af0-5bb83d3f1468 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:35.649041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49476d1-a492-416a-8af0-5bb83d3f1468 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:35.654173 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e1523-97d6-4282-a5f9-92aa480284ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:35.654940 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e1523-97d6-4282-a5f9-92aa480284ea admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:35.655219 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e1523-97d6-4282-a5f9-92aa480284ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:35.655528 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e1523-97d6-4282-a5f9-92aa480284ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:35.656523 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e1523-97d6-4282-a5f9-92aa480284ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80255) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:57:35.656811 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e1523-97d6-4282-a5f9-92aa480284ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:35.658665 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e1523-97d6-4282-a5f9-92aa480284ea admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:35.670894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1411/2809] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:57:35.675616 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8103d-292c-43ec-9614-7f43d6c46f05 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:35.685496 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8103d-292c-43ec-9614-7f43d6c46f05 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:35.690260 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fba1b8-cc34-44d2-af39-602e4c098a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:35.690928 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fba1b8-cc34-44d2-af39-602e4c098a06 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:35.691798 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fba1b8-cc34-44d2-af39-602e4c098a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:35.692575 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fba1b8-cc34-44d2-af39-602e4c098a06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:35.693260 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fba1b8-cc34-44d2-af39-602e4c098a06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:35.695208 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fba1b8-cc34-44d2-af39-602e4c098a06 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:36.268698 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1399/2810] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:35 2022] POST /identity/v3/projects => generated 377 bytes in 593 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:36.273048 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e1523-97d6-4282-a5f9-92aa480284ea admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:36.284693 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e1523-97d6-4282-a5f9-92aa480284ea admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:36.349707 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce2a3a-9cbd-46e1-b67a-15f3a6d19166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:36.350641 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce2a3a-9cbd-46e1-b67a-15f3a6d19166 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:36.350973 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce2a3a-9cbd-46e1-b67a-15f3a6d19166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:36.351257 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce2a3a-9cbd-46e1-b67a-15f3a6d19166 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:36.351947 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce2a3a-9cbd-46e1-b67a-15f3a6d19166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:36.354564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce2a3a-9cbd-46e1-b67a-15f3a6d19166 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:36.427447 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1412/2811] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:36 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:36.431744 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fba1b8-cc34-44d2-af39-602e4c098a06 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:36.444725 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fba1b8-cc34-44d2-af39-602e4c098a06 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:36.452838 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d007c-b373-4185-ab48-00c062a79bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:36.453888 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d007c-b373-4185-ab48-00c062a79bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:36.454876 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d007c-b373-4185-ab48-00c062a79bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:36.455326 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d007c-b373-4185-ab48-00c062a79bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:36.456262 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d007c-b373-4185-ab48-00c062a79bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:36.458015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d007c-b373-4185-ab48-00c062a79bd2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:36.476359 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1400/2812] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:36.477003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce2a3a-9cbd-46e1-b67a-15f3a6d19166 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:36.487498 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce2a3a-9cbd-46e1-b67a-15f3a6d19166 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:36.495271 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a062d76-9b43-4889-bf35-9dbf50b9721d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:36.495271 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a062d76-9b43-4889-bf35-9dbf50b9721d admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:36.495544 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a062d76-9b43-4889-bf35-9dbf50b9721d admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:36.495801 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a062d76-9b43-4889-bf35-9dbf50b9721d admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:36.496495 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a062d76-9b43-4889-bf35-9dbf50b9721d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:36.498089 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a062d76-9b43-4889-bf35-9dbf50b9721d admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:36.510589 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1413/2813] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:36.513976 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d007c-b373-4185-ab48-00c062a79bd2 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:36.524039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d007c-b373-4185-ab48-00c062a79bd2 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:36.528646 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892c48b-d463-4dd5-b9ff-17135a2975e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:36.529520 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892c48b-d463-4dd5-b9ff-17135a2975e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:36.529803 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892c48b-d463-4dd5-b9ff-17135a2975e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:36.530077 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892c48b-d463-4dd5-b9ff-17135a2975e1 admin admin] PATH_INFO: `/v3/projects/b9d6d8ea4e304532a993d0fd330eb5d7/users/cdd9ef6fe97f471c8912f4ccf31ba045/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:36.545668 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892c48b-d463-4dd5-b9ff-17135a2975e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9d6d8ea4e304532a993d0fd330eb5d7, user_id=cdd9ef6fe97f471c8912f4ccf31ba045, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:36.549248 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892c48b-d463-4dd5-b9ff-17135a2975e1 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:36.653971 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1401/2814] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:36 2022] PUT /identity/v3/projects/b9d6d8ea4e304532a993d0fd330eb5d7/users/cdd9ef6fe97f471c8912f4ccf31ba045/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:36.665206 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d00fc7-4410-43a5-b7b9-01cac8682a67 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:36.665735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d00fc7-4410-43a5-b7b9-01cac8682a67 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:36.665735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d00fc7-4410-43a5-b7b9-01cac8682a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:36.695433 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-19d00fc7-4410-43a5-b7b9-01cac8682a67 None None] MFA Rules not processed for user `cdd9ef6fe97f471c8912f4ccf31ba045`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:36.696639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-19d00fc7-4410-43a5-b7b9-01cac8682a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:36.788520 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1414/2815] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:36.788648 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892c48b-d463-4dd5-b9ff-17135a2975e1 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:36.836398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:36.852202 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892c48b-d463-4dd5-b9ff-17135a2975e1 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:36.857539 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99903f-c5aa-4da4-9a51-0dbe1d957964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:36.858210 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99903f-c5aa-4da4-9a51-0dbe1d957964 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:36.858497 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99903f-c5aa-4da4-9a51-0dbe1d957964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:36.858835 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99903f-c5aa-4da4-9a51-0dbe1d957964 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:36.859539 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99903f-c5aa-4da4-9a51-0dbe1d957964 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:36.861480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99903f-c5aa-4da4-9a51-0dbe1d957964 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:36.976187 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1402/2816] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:36 2022] POST /identity/v3/projects => generated 377 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:36.980670 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d00fc7-4410-43a5-b7b9-01cac8682a67 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:36.992002 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d00fc7-4410-43a5-b7b9-01cac8682a67 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:36.996187 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:37.052108 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8adc2-ba43-4f89-99e6-6c5abc01e2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:37.052838 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8adc2-ba43-4f89-99e6-6c5abc01e2c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:37.053081 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8adc2-ba43-4f89-99e6-6c5abc01e2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:37.053357 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8adc2-ba43-4f89-99e6-6c5abc01e2c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:37.054067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8adc2-ba43-4f89-99e6-6c5abc01e2c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:37.056055 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8adc2-ba43-4f89-99e6-6c5abc01e2c7 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:37.139410 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1415/2817] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:36 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:37.143548 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99903f-c5aa-4da4-9a51-0dbe1d957964 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:37.155496 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99903f-c5aa-4da4-9a51-0dbe1d957964 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:37.162078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8684d-2704-4fb5-bc16-aa5494e9f84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:37.162796 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8684d-2704-4fb5-bc16-aa5494e9f84f admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:37.163494 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8684d-2704-4fb5-bc16-aa5494e9f84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:37.163494 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8684d-2704-4fb5-bc16-aa5494e9f84f admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:37.164116 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8684d-2704-4fb5-bc16-aa5494e9f84f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:37.165946 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8684d-2704-4fb5-bc16-aa5494e9f84f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:37.184039 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8adc2-ba43-4f89-99e6-6c5abc01e2c7 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:37.188618 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1403/2818] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:37.196821 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8adc2-ba43-4f89-99e6-6c5abc01e2c7 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:37.202650 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86c776-9c28-4483-bb87-b0b2fb728865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:37.203418 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86c776-9c28-4483-bb87-b0b2fb728865 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:37.203775 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86c776-9c28-4483-bb87-b0b2fb728865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:37.204123 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86c776-9c28-4483-bb87-b0b2fb728865 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:37.205580 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce86c776-9c28-4483-bb87-b0b2fb728865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:37.212231 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce86c776-9c28-4483-bb87-b0b2fb728865 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:37.224894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1416/2819] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:37.229048 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8684d-2704-4fb5-bc16-aa5494e9f84f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:37.240682 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8684d-2704-4fb5-bc16-aa5494e9f84f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:37.246129 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed4cb9-f5bc-4c3f-bba3-69aa0c89c717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:37.247272 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed4cb9-f5bc-4c3f-bba3-69aa0c89c717 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:37.247667 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed4cb9-f5bc-4c3f-bba3-69aa0c89c717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:37.248066 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed4cb9-f5bc-4c3f-bba3-69aa0c89c717 admin admin] PATH_INFO: `/v3/projects/12ea419a46dc4ac39e2c32c839cd9692/users/5c663297b4f144f285e9d5eea2ef295e/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:37.266298 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed4cb9-f5bc-4c3f-bba3-69aa0c89c717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12ea419a46dc4ac39e2c32c839cd9692, user_id=5c663297b4f144f285e9d5eea2ef295e, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:37.274693 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed4cb9-f5bc-4c3f-bba3-69aa0c89c717 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:37.357131 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1404/2820] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:37 2022] PUT /identity/v3/projects/12ea419a46dc4ac39e2c32c839cd9692/users/5c663297b4f144f285e9d5eea2ef295e/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:37.369334 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7ce24-13df-49f0-9c97-2d087a4b85a6 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:37.369709 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7ce24-13df-49f0-9c97-2d087a4b85a6 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:37.370313 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7ce24-13df-49f0-9c97-2d087a4b85a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:37.406010 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-a6b7ce24-13df-49f0-9c97-2d087a4b85a6 None None] MFA Rules not processed for user `5c663297b4f144f285e9d5eea2ef295e`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:37.407261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a6b7ce24-13df-49f0-9c97-2d087a4b85a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:37.482833 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1417/2821] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:37.490656 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed4cb9-f5bc-4c3f-bba3-69aa0c89c717 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:37.559460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed4cb9-f5bc-4c3f-bba3-69aa0c89c717 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:37.564562 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a945bf-8881-41cf-822f-5b668a286a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:37.565230 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a945bf-8881-41cf-822f-5b668a286a56 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:37.565574 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a945bf-8881-41cf-822f-5b668a286a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:37.566010 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a945bf-8881-41cf-822f-5b668a286a56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:37.567676 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a945bf-8881-41cf-822f-5b668a286a56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:37.569609 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a945bf-8881-41cf-822f-5b668a286a56 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:37.696310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1405/2822] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:37 2022] POST /identity/v3/projects => generated 377 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:37.700273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7ce24-13df-49f0-9c97-2d087a4b85a6 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:37.710852 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7ce24-13df-49f0-9c97-2d087a4b85a6 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:37.783369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c443cd8-9a9e-4ec9-a41a-b58ce34ce782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:37.784177 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c443cd8-9a9e-4ec9-a41a-b58ce34ce782 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:37.784525 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c443cd8-9a9e-4ec9-a41a-b58ce34ce782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:37.784873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c443cd8-9a9e-4ec9-a41a-b58ce34ce782 admin admin] PATH_INFO: `/v3/users` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:37.786599 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c443cd8-9a9e-4ec9-a41a-b58ce34ce782 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:37.788539 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c443cd8-9a9e-4ec9-a41a-b58ce34ce782 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:37.900222 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1418/2823] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:37 2022] POST /identity/v3/users => generated 295 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:37.909039 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a945bf-8881-41cf-822f-5b668a286a56 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:37.924825 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a945bf-8881-41cf-822f-5b668a286a56 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:37.935401 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791140c-e514-4d21-af97-b40985e16ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:37.936647 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791140c-e514-4d21-af97-b40985e16ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:37.939264 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791140c-e514-4d21-af97-b40985e16ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:37.939264 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791140c-e514-4d21-af97-b40985e16ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:37.939264 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791140c-e514-4d21-af97-b40985e16ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:37.940727 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791140c-e514-4d21-af97-b40985e16ee5 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:37.967348 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1406/2824] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:37.972493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c443cd8-9a9e-4ec9-a41a-b58ce34ce782 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:37.986071 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:37.995522 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c443cd8-9a9e-4ec9-a41a-b58ce34ce782 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:38.006674 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bf30f-3fd4-4204-820f-0b93ad58943a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:38.008004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bf30f-3fd4-4204-820f-0b93ad58943a admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:38.008539 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bf30f-3fd4-4204-820f-0b93ad58943a admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:38.009022 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bf30f-3fd4-4204-820f-0b93ad58943a admin admin] PATH_INFO: `/v3/projects/d991d7c4d4a34114ba071eda025bc11c/users/ae44a59ec0724297bdc3c53597b05b3b/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:38.048198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bf30f-3fd4-4204-820f-0b93ad58943a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d991d7c4d4a34114ba071eda025bc11c, user_id=ae44a59ec0724297bdc3c53597b05b3b, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:38.051693 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bf30f-3fd4-4204-820f-0b93ad58943a admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:38.152799 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1419/2825] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:37 2022] PUT /identity/v3/projects/d991d7c4d4a34114ba071eda025bc11c/users/ae44a59ec0724297bdc3c53597b05b3b/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:38.156692 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791140c-e514-4d21-af97-b40985e16ee5 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:38.184992 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791140c-e514-4d21-af97-b40985e16ee5 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:38.272516 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacdf7b-5708-4382-a82e-3e8aa2213e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:38.273305 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacdf7b-5708-4382-a82e-3e8aa2213e9e admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:38.274265 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacdf7b-5708-4382-a82e-3e8aa2213e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:38.274265 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacdf7b-5708-4382-a82e-3e8aa2213e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:38.275069 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eacdf7b-5708-4382-a82e-3e8aa2213e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:38.276784 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eacdf7b-5708-4382-a82e-3e8aa2213e9e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:38.288709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1407/2826] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:38.292693 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bf30f-3fd4-4204-820f-0b93ad58943a admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:38.303414 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bf30f-3fd4-4204-820f-0b93ad58943a admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:38.309297 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d4f94-3f88-4138-9c87-8920937468ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:38.310509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d4f94-3f88-4138-9c87-8920937468ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:38.310509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d4f94-3f88-4138-9c87-8920937468ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:38.310816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d4f94-3f88-4138-9c87-8920937468ba admin admin] PATH_INFO: `/v3/projects/d991d7c4d4a34114ba071eda025bc11c/users/ae44a59ec0724297bdc3c53597b05b3b/roles/c9e006de0d6c4812ab29d778e9c8a830` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:38.312804 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d4f94-3f88-4138-9c87-8920937468ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=d991d7c4d4a34114ba071eda025bc11c, user_id=ae44a59ec0724297bdc3c53597b05b3b, role_id=c9e006de0d6c4812ab29d778e9c8a830)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:38.315480 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d4f94-3f88-4138-9c87-8920937468ba admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:38.326298 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1420/2827] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:38 2022] PUT /identity/v3/projects/d991d7c4d4a34114ba071eda025bc11c/users/ae44a59ec0724297bdc3c53597b05b3b/roles/c9e006de0d6c4812ab29d778e9c8a830 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:38.336991 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addeb47b-4424-4839-9b08-28f98e654420 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:38.337403 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addeb47b-4424-4839-9b08-28f98e654420 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:38.337699 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addeb47b-4424-4839-9b08-28f98e654420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:38.351111 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:38.366966 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-addeb47b-4424-4839-9b08-28f98e654420 None None] MFA Rules not processed for user `ae44a59ec0724297bdc3c53597b05b3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:38.368256 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-addeb47b-4424-4839-9b08-28f98e654420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:38.467365 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1408/2828] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:38 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:38.479036 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d4f94-3f88-4138-9c87-8920937468ba admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:38.480735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-0c8d4f94-3f88-4138-9c87-8920937468ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:38.555823 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d4f94-3f88-4138-9c87-8920937468ba admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:38.559994 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1876c954-93d5-4f9c-82b0-0aa6293099d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:38.560800 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876c954-93d5-4f9c-82b0-0aa6293099d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:38.561168 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876c954-93d5-4f9c-82b0-0aa6293099d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:38.561505 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876c954-93d5-4f9c-82b0-0aa6293099d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:38.635633 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addeb47b-4424-4839-9b08-28f98e654420 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:38.641023 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-addeb47b-4424-4839-9b08-28f98e654420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:38.645070 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1876c954-93d5-4f9c-82b0-0aa6293099d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:38.646894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1876c954-93d5-4f9c-82b0-0aa6293099d5 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:38.657108 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1421/2829] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:38.718365 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addeb47b-4424-4839-9b08-28f98e654420 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:38.725243 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d674c-907c-4e7a-aa4b-9989ffc078ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:38.726977 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d674c-907c-4e7a-aa4b-9989ffc078ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:38.727240 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d674c-907c-4e7a-aa4b-9989ffc078ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:38.727514 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d674c-907c-4e7a-aa4b-9989ffc078ef admin admin] PATH_INFO: `/v3/users/a673ea6aebc241fb9c9f5037c4d72b0e` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:38.729105 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d674c-907c-4e7a-aa4b-9989ffc078ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=a673ea6aebc241fb9c9f5037c4d72b0e)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:38.731352 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d674c-907c-4e7a-aa4b-9989ffc078ef admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:39.015939 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f11d674c-907c-4e7a-aa4b-9989ffc078ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a673ea6aebc241fb9c9f5037c4d72b0e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:39.023823 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f11d674c-907c-4e7a-aa4b-9989ffc078ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a673ea6aebc241fb9c9f5037c4d72b0e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:39.024556 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f11d674c-907c-4e7a-aa4b-9989ffc078ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a673ea6aebc241fb9c9f5037c4d72b0e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:39.025225 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f11d674c-907c-4e7a-aa4b-9989ffc078ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a673ea6aebc241fb9c9f5037c4d72b0e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:39.038979 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f11d674c-907c-4e7a-aa4b-9989ffc078ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a673ea6aebc241fb9c9f5037c4d72b0e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:39.130069 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1409/2830] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:38 2022] DELETE /identity/v3/users/a673ea6aebc241fb9c9f5037c4d72b0e => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:39.134884 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0d78e-918f-4ceb-b3e2-6a8db019e3c7 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:39.135378 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0d78e-918f-4ceb-b3e2-6a8db019e3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:39.135773 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0d78e-918f-4ceb-b3e2-6a8db019e3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:39.155845 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-c5b0d78e-918f-4ceb-b3e2-6a8db019e3c7 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:39.161036 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c5b0d78e-918f-4ceb-b3e2-6a8db019e3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:39.221873 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1422/2831] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:39.236104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d674c-907c-4e7a-aa4b-9989ffc078ef admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:39.236104 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f11d674c-907c-4e7a-aa4b-9989ffc078ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:39.251921 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:39.319874 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d674c-907c-4e7a-aa4b-9989ffc078ef admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:39.323363 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b57ce1-3275-406e-9a6a-b6a9ca1eeb3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:39.324107 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b57ce1-3275-406e-9a6a-b6a9ca1eeb3c service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:39.325041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b57ce1-3275-406e-9a6a-b6a9ca1eeb3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:39.325041 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b57ce1-3275-406e-9a6a-b6a9ca1eeb3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:39.335815 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b57ce1-3275-406e-9a6a-b6a9ca1eeb3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:39.337841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b57ce1-3275-406e-9a6a-b6a9ca1eeb3c service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:39.350951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1410/2832] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:39.809593 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0d78e-918f-4ceb-b3e2-6a8db019e3c7 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:39.818042 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-c5b0d78e-918f-4ceb-b3e2-6a8db019e3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:39.824406 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0d78e-918f-4ceb-b3e2-6a8db019e3c7 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:39.829191 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff044e-9dd2-4490-9534-506151cccca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:39.830177 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff044e-9dd2-4490-9534-506151cccca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:39.830630 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff044e-9dd2-4490-9534-506151cccca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:39.831060 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff044e-9dd2-4490-9534-506151cccca6 admin admin] PATH_INFO: `/v3/projects/d6e6f7e0d03a4e968775df1aa547cfbf` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:39.833509 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff044e-9dd2-4490-9534-506151cccca6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6e6f7e0d03a4e968775df1aa547cfbf)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:39.835580 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff044e-9dd2-4490-9534-506151cccca6 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:39.855302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:39.976287 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-25ff044e-9dd2-4490-9534-506151cccca6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6e6f7e0d03a4e968775df1aa547cfbf', 'request_id': 'req-25ff044e-9dd2-4490-9534-506151cccca6'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:39.983193 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-25ff044e-9dd2-4490-9534-506151cccca6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6e6f7e0d03a4e968775df1aa547cfbf', 'request_id': 'req-25ff044e-9dd2-4490-9534-506151cccca6'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:40.063648 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-25ff044e-9dd2-4490-9534-506151cccca6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6e6f7e0d03a4e968775df1aa547cfbf', 'request_id': 'req-25ff044e-9dd2-4490-9534-506151cccca6'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:40.076269 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-25ff044e-9dd2-4490-9534-506151cccca6 admin admin] The token cache is being invalidate because project d6e6f7e0d03a4e968775df1aa547cfbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:40.076791 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-25ff044e-9dd2-4490-9534-506151cccca6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:40.077554 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-25ff044e-9dd2-4490-9534-506151cccca6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:40.083880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1423/2833] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:39 2022] DELETE /identity/v3/projects/d6e6f7e0d03a4e968775df1aa547cfbf => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:40.085121 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6429caa-a734-4f0c-a856-22ca3c17b754 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:40.085603 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6429caa-a734-4f0c-a856-22ca3c17b754 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:40.085690 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6429caa-a734-4f0c-a856-22ca3c17b754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:40.105510 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b6429caa-a734-4f0c-a856-22ca3c17b754 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:40.107271 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b6429caa-a734-4f0c-a856-22ca3c17b754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:40.177622 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1411/2834] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:40.185554 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7110a19-9f31-46f0-a6c5-dbfbdf51206b None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:40.186068 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7110a19-9f31-46f0-a6c5-dbfbdf51206b None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:40.186289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7110a19-9f31-46f0-a6c5-dbfbdf51206b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:40.205691 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-b7110a19-9f31-46f0-a6c5-dbfbdf51206b None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:40.207117 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-b7110a19-9f31-46f0-a6c5-dbfbdf51206b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:40.215216 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1424/2835] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:40.219536 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6429caa-a734-4f0c-a856-22ca3c17b754 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:40.229709 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6429caa-a734-4f0c-a856-22ca3c17b754 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:40.234666 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66038f7e-3c1d-4a1c-809d-697d11487def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:40.235319 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66038f7e-3c1d-4a1c-809d-697d11487def admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:40.235606 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66038f7e-3c1d-4a1c-809d-697d11487def admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:40.235880 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66038f7e-3c1d-4a1c-809d-697d11487def admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:40.236601 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66038f7e-3c1d-4a1c-809d-697d11487def admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:57:40.236913 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66038f7e-3c1d-4a1c-809d-697d11487def admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:40.239112 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66038f7e-3c1d-4a1c-809d-697d11487def admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:40.250775 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1412/2836] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:57:40.255791 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7110a19-9f31-46f0-a6c5-dbfbdf51206b None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:40.265761 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7110a19-9f31-46f0-a6c5-dbfbdf51206b None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:40.270553 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62a496-02f1-4878-83bd-d321c0f741f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:40.274027 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62a496-02f1-4878-83bd-d321c0f741f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:40.274027 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62a496-02f1-4878-83bd-d321c0f741f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:40.274027 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62a496-02f1-4878-83bd-d321c0f741f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:40.274027 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62a496-02f1-4878-83bd-d321c0f741f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:40.274425 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62a496-02f1-4878-83bd-d321c0f741f6 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:40.484481 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1425/2837] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:40 2022] POST /identity/v3/projects => generated 397 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:40.490124 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66038f7e-3c1d-4a1c-809d-697d11487def admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:40.501294 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66038f7e-3c1d-4a1c-809d-697d11487def admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:40.562357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226b4e7-07bd-483b-ae72-c3026998f998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:40.563078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226b4e7-07bd-483b-ae72-c3026998f998 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:40.563365 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226b4e7-07bd-483b-ae72-c3026998f998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:40.563640 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226b4e7-07bd-483b-ae72-c3026998f998 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:40.564330 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226b4e7-07bd-483b-ae72-c3026998f998 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:40.566339 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226b4e7-07bd-483b-ae72-c3026998f998 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:40.694703 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62a496-02f1-4878-83bd-d321c0f741f6 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:40.696494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-7d62a496-02f1-4878-83bd-d321c0f741f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:40.710420 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1413/2838] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:40 2022] POST /identity/v3/users => generated 299 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:40.723631 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226b4e7-07bd-483b-ae72-c3026998f998 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:40.734492 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226b4e7-07bd-483b-ae72-c3026998f998 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:40.740635 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890d513-7d4d-453e-b1b0-583330a56c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:40.741320 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890d513-7d4d-453e-b1b0-583330a56c9b admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:40.741605 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890d513-7d4d-453e-b1b0-583330a56c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:40.741879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890d513-7d4d-453e-b1b0-583330a56c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:40.742655 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b890d513-7d4d-453e-b1b0-583330a56c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:40.744476 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b890d513-7d4d-453e-b1b0-583330a56c9b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:40.761200 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1414/2839] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:40.765408 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62a496-02f1-4878-83bd-d321c0f741f6 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:40.768379 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890d513-7d4d-453e-b1b0-583330a56c9b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:40.769203 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba443098-76e6-4db8-9a46-b3127d8b7ae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:40.770141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba443098-76e6-4db8-9a46-b3127d8b7ae7 service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:40.770429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba443098-76e6-4db8-9a46-b3127d8b7ae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:40.771808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba443098-76e6-4db8-9a46-b3127d8b7ae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:40.773695 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-ba443098-76e6-4db8-9a46-b3127d8b7ae7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:40.780074 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890d513-7d4d-453e-b1b0-583330a56c9b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:40.787705 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a2bce-0ebf-4821-882c-213bd6cdc6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:40.788407 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a2bce-0ebf-4821-882c-213bd6cdc6ca admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:40.788754 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a2bce-0ebf-4821-882c-213bd6cdc6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:40.788957 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a2bce-0ebf-4821-882c-213bd6cdc6ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:40.789723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a2bce-0ebf-4821-882c-213bd6cdc6ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:40.791633 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a2bce-0ebf-4821-882c-213bd6cdc6ca admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:40.804357 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1415/2840] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:40.808529 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a2bce-0ebf-4821-882c-213bd6cdc6ca admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:40.823422 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a2bce-0ebf-4821-882c-213bd6cdc6ca admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:40.828723 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923eb75-a027-42c2-94e2-2cf114811930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:40.829685 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923eb75-a027-42c2-94e2-2cf114811930 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:40.830088 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923eb75-a027-42c2-94e2-2cf114811930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:40.830417 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923eb75-a027-42c2-94e2-2cf114811930 admin admin] PATH_INFO: `/v3/projects/c7e018ee10d940868440d1fc624228d8/users/55a7bbfd2a3e4660bc1acdf6be6513e1/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:40.834330 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:40.844808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba443098-76e6-4db8-9a46-b3127d8b7ae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:40.846298 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba443098-76e6-4db8-9a46-b3127d8b7ae7 service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:40.847163 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923eb75-a027-42c2-94e2-2cf114811930 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7e018ee10d940868440d1fc624228d8, user_id=55a7bbfd2a3e4660bc1acdf6be6513e1, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:40.850038 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923eb75-a027-42c2-94e2-2cf114811930 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:40.858070 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1426/2841] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:40.934099 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1416/2842] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:40 2022] PUT /identity/v3/projects/c7e018ee10d940868440d1fc624228d8/users/55a7bbfd2a3e4660bc1acdf6be6513e1/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:40.946723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7e73f-edc0-4efc-9de8-8502c5ec9c59 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:40.947016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7e73f-edc0-4efc-9de8-8502c5ec9c59 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:40.947298 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7e73f-edc0-4efc-9de8-8502c5ec9c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:40.975717 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-34e7e73f-edc0-4efc-9de8-8502c5ec9c59 None None] MFA Rules not processed for user `55a7bbfd2a3e4660bc1acdf6be6513e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:40.976922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-34e7e73f-edc0-4efc-9de8-8502c5ec9c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:40.996519 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:41.042141 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1427/2843] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:41.054428 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923eb75-a027-42c2-94e2-2cf114811930 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:41.056470 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9923eb75-a027-42c2-94e2-2cf114811930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:41.113113 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923eb75-a027-42c2-94e2-2cf114811930 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:41.117415 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf47f5-c680-421b-b83c-65680ebd8500 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:41.118063 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf47f5-c680-421b-b83c-65680ebd8500 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:41.118350 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf47f5-c680-421b-b83c-65680ebd8500 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:41.118661 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf47f5-c680-421b-b83c-65680ebd8500 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:41.126972 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf47f5-c680-421b-b83c-65680ebd8500 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:41.128413 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf47f5-c680-421b-b83c-65680ebd8500 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:41.137657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1417/2844] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:41 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:42.579158 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7e73f-edc0-4efc-9de8-8502c5ec9c59 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:42.588728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7e73f-edc0-4efc-9de8-8502c5ec9c59 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:42.592911 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc16c3-73a5-48cc-a966-cd531451cf2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:42.593572 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc16c3-73a5-48cc-a966-cd531451cf2b service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:42.593920 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc16c3-73a5-48cc-a966-cd531451cf2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:42.594132 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc16c3-73a5-48cc-a966-cd531451cf2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:42.595922 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-36dc16c3-73a5-48cc-a966-cd531451cf2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:42.658635 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc16c3-73a5-48cc-a966-cd531451cf2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:42.660908 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc16c3-73a5-48cc-a966-cd531451cf2b service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:42.670695 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1428/2845] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:51.350752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=32,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:52.666113 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=48 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:52.895962 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf47f5-c680-421b-b83c-65680ebd8500 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:52.960211 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf47f5-c680-421b-b83c-65680ebd8500 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:52.964872 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38cd84-5d6d-41e0-969f-6c6d071ffa9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:52.965754 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38cd84-5d6d-41e0-969f-6c6d071ffa9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:52.966032 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38cd84-5d6d-41e0-969f-6c6d071ffa9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:52.966915 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38cd84-5d6d-41e0-969f-6c6d071ffa9f admin admin] PATH_INFO: `/v3/users/55a7bbfd2a3e4660bc1acdf6be6513e1` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:52.967589 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38cd84-5d6d-41e0-969f-6c6d071ffa9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=55a7bbfd2a3e4660bc1acdf6be6513e1)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:52.969522 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38cd84-5d6d-41e0-969f-6c6d071ffa9f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:53.157033 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0f38cd84-5d6d-41e0-969f-6c6d071ffa9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55a7bbfd2a3e4660bc1acdf6be6513e1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:53.165211 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0f38cd84-5d6d-41e0-969f-6c6d071ffa9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55a7bbfd2a3e4660bc1acdf6be6513e1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:53.165893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0f38cd84-5d6d-41e0-969f-6c6d071ffa9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55a7bbfd2a3e4660bc1acdf6be6513e1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:53.166620 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0f38cd84-5d6d-41e0-969f-6c6d071ffa9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55a7bbfd2a3e4660bc1acdf6be6513e1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:53.179793 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0f38cd84-5d6d-41e0-969f-6c6d071ffa9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55a7bbfd2a3e4660bc1acdf6be6513e1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:53.233996 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc16c3-73a5-48cc-a966-cd531451cf2b service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:53.235568 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-36dc16c3-73a5-48cc-a966-cd531451cf2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:53.249245 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc16c3-73a5-48cc-a966-cd531451cf2b service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:53.253848 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6da01b-28d5-4566-bfda-7a617e91f315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:53.254733 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6da01b-28d5-4566-bfda-7a617e91f315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:53.255019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6da01b-28d5-4566-bfda-7a617e91f315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:53.255293 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6da01b-28d5-4566-bfda-7a617e91f315 admin admin] PATH_INFO: `/v3/users/cdd9ef6fe97f471c8912f4ccf31ba045` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:53.256669 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6da01b-28d5-4566-bfda-7a617e91f315 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdd9ef6fe97f471c8912f4ccf31ba045)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:53.258646 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6da01b-28d5-4566-bfda-7a617e91f315 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:53.265070 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1418/2846] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:52 2022] DELETE /identity/v3/users/55a7bbfd2a3e4660bc1acdf6be6513e1 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:53.269465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06547e4e-c9fa-4927-b226-79940ae7b746 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:53.269825 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06547e4e-c9fa-4927-b226-79940ae7b746 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:53.270184 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06547e4e-c9fa-4927-b226-79940ae7b746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:53.290093 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-06547e4e-c9fa-4927-b226-79940ae7b746 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:53.291536 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-06547e4e-c9fa-4927-b226-79940ae7b746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:53.352686 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1419/2847] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:53.362287 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06547e4e-c9fa-4927-b226-79940ae7b746 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:53.363687 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-06547e4e-c9fa-4927-b226-79940ae7b746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:53.385805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:53.423907 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06547e4e-c9fa-4927-b226-79940ae7b746 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:53.427720 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b472e-4ad0-42ef-84bc-b03d04b92760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:53.428426 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b472e-4ad0-42ef-84bc-b03d04b92760 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:53.428773 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b472e-4ad0-42ef-84bc-b03d04b92760 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:53.429050 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b472e-4ad0-42ef-84bc-b03d04b92760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:53.438641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b472e-4ad0-42ef-84bc-b03d04b92760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:53.440292 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b472e-4ad0-42ef-84bc-b03d04b92760 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:53.451592 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1420/2848] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:57:53.487200 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-1e6da01b-28d5-4566-bfda-7a617e91f315 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdd9ef6fe97f471c8912f4ccf31ba045'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:53.494844 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-1e6da01b-28d5-4566-bfda-7a617e91f315 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdd9ef6fe97f471c8912f4ccf31ba045'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:53.495689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-1e6da01b-28d5-4566-bfda-7a617e91f315 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdd9ef6fe97f471c8912f4ccf31ba045'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:53.496380 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-1e6da01b-28d5-4566-bfda-7a617e91f315 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdd9ef6fe97f471c8912f4ccf31ba045'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:53.518942 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-1e6da01b-28d5-4566-bfda-7a617e91f315 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdd9ef6fe97f471c8912f4ccf31ba045'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:53.617484 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1429/2849] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:53 2022] DELETE /identity/v3/users/cdd9ef6fe97f471c8912f4ccf31ba045 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:53.622023 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1d69a-1f85-4fb0-aebf-671f39179681 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:53.622489 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1d69a-1f85-4fb0-aebf-671f39179681 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:53.622833 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1d69a-1f85-4fb0-aebf-671f39179681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:53.640767 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-42b1d69a-1f85-4fb0-aebf-671f39179681 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:53.642451 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-42b1d69a-1f85-4fb0-aebf-671f39179681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:53.706120 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1421/2850] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:57:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:53.740728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6da01b-28d5-4566-bfda-7a617e91f315 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:53.742423 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-1e6da01b-28d5-4566-bfda-7a617e91f315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:53.802874 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6da01b-28d5-4566-bfda-7a617e91f315 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:53.807035 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244cb9a-ca9a-4ff5-8468-8d3a1b7cc39b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:53.816678 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244cb9a-ca9a-4ff5-8468-8d3a1b7cc39b service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:53.816678 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244cb9a-ca9a-4ff5-8468-8d3a1b7cc39b service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:53.816678 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244cb9a-ca9a-4ff5-8468-8d3a1b7cc39b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:53.819754 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f244cb9a-ca9a-4ff5-8468-8d3a1b7cc39b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:53.821332 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f244cb9a-ca9a-4ff5-8468-8d3a1b7cc39b service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:53.840343 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1430/2851] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:57:53.913220 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1d69a-1f85-4fb0-aebf-671f39179681 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:53.915092 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-42b1d69a-1f85-4fb0-aebf-671f39179681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:53.940696 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1d69a-1f85-4fb0-aebf-671f39179681 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:53.946980 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507406b-9aa0-4dac-a530-a5b67b884dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:53.947923 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507406b-9aa0-4dac-a530-a5b67b884dbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:53.948308 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507406b-9aa0-4dac-a530-a5b67b884dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:53.948623 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507406b-9aa0-4dac-a530-a5b67b884dbd admin admin] PATH_INFO: `/v3/projects/c7e018ee10d940868440d1fc624228d8` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:53.950524 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507406b-9aa0-4dac-a530-a5b67b884dbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7e018ee10d940868440d1fc624228d8)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:53.952559 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507406b-9aa0-4dac-a530-a5b67b884dbd admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:54.092896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2507406b-9aa0-4dac-a530-a5b67b884dbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7e018ee10d940868440d1fc624228d8', 'request_id': 'req-2507406b-9aa0-4dac-a530-a5b67b884dbd'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:54.100281 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2507406b-9aa0-4dac-a530-a5b67b884dbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7e018ee10d940868440d1fc624228d8', 'request_id': 'req-2507406b-9aa0-4dac-a530-a5b67b884dbd'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:54.201887 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2507406b-9aa0-4dac-a530-a5b67b884dbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7e018ee10d940868440d1fc624228d8', 'request_id': 'req-2507406b-9aa0-4dac-a530-a5b67b884dbd'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:54.213453 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2507406b-9aa0-4dac-a530-a5b67b884dbd admin admin] The token cache is being invalidate because project c7e018ee10d940868440d1fc624228d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:54.213741 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2507406b-9aa0-4dac-a530-a5b67b884dbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:54.214365 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2507406b-9aa0-4dac-a530-a5b67b884dbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:54.215937 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1422/2852] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:53 2022] DELETE /identity/v3/projects/c7e018ee10d940868440d1fc624228d8 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:54.306493 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244cb9a-ca9a-4ff5-8468-8d3a1b7cc39b service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:54.310062 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f244cb9a-ca9a-4ff5-8468-8d3a1b7cc39b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:54.354744 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244cb9a-ca9a-4ff5-8468-8d3a1b7cc39b service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:54.450564 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474471c-181b-412e-bea5-5c0b9e984f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:54.451637 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474471c-181b-412e-bea5-5c0b9e984f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:54.451923 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474471c-181b-412e-bea5-5c0b9e984f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:54.452205 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474471c-181b-412e-bea5-5c0b9e984f85 admin admin] PATH_INFO: `/v3/projects/b9d6d8ea4e304532a993d0fd330eb5d7` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:54.453598 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474471c-181b-412e-bea5-5c0b9e984f85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9d6d8ea4e304532a993d0fd330eb5d7)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:54.455950 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474471c-181b-412e-bea5-5c0b9e984f85 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:54.503764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:54.599346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4474471c-181b-412e-bea5-5c0b9e984f85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9d6d8ea4e304532a993d0fd330eb5d7', 'request_id': 'req-4474471c-181b-412e-bea5-5c0b9e984f85'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:54.607741 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4474471c-181b-412e-bea5-5c0b9e984f85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9d6d8ea4e304532a993d0fd330eb5d7', 'request_id': 'req-4474471c-181b-412e-bea5-5c0b9e984f85'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:54.687848 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4474471c-181b-412e-bea5-5c0b9e984f85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9d6d8ea4e304532a993d0fd330eb5d7', 'request_id': 'req-4474471c-181b-412e-bea5-5c0b9e984f85'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:54.699434 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4474471c-181b-412e-bea5-5c0b9e984f85 admin admin] The token cache is being invalidate because project b9d6d8ea4e304532a993d0fd330eb5d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:54.699789 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4474471c-181b-412e-bea5-5c0b9e984f85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:54.700353 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-4474471c-181b-412e-bea5-5c0b9e984f85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:54.701889 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1431/2853] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:54 2022] DELETE /identity/v3/projects/b9d6d8ea4e304532a993d0fd330eb5d7 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:54.705763 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507406b-9aa0-4dac-a530-a5b67b884dbd admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:54.707165 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2507406b-9aa0-4dac-a530-a5b67b884dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:54.720444 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507406b-9aa0-4dac-a530-a5b67b884dbd admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:54.772749 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:54.791234 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933a658-b1af-4ee1-ab84-f3ea3a12b0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:54.792234 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933a658-b1af-4ee1-ab84-f3ea3a12b0ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:54.792612 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933a658-b1af-4ee1-ab84-f3ea3a12b0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:54.792972 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933a658-b1af-4ee1-ab84-f3ea3a12b0ad admin admin] PATH_INFO: `/v3/users/5c663297b4f144f285e9d5eea2ef295e` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:54.794333 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933a658-b1af-4ee1-ab84-f3ea3a12b0ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c663297b4f144f285e9d5eea2ef295e)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:54.796509 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933a658-b1af-4ee1-ab84-f3ea3a12b0ad admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:54.962864 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0933a658-b1af-4ee1-ab84-f3ea3a12b0ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c663297b4f144f285e9d5eea2ef295e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:54.969474 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0933a658-b1af-4ee1-ab84-f3ea3a12b0ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c663297b4f144f285e9d5eea2ef295e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:54.970144 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0933a658-b1af-4ee1-ab84-f3ea3a12b0ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c663297b4f144f285e9d5eea2ef295e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:54.970830 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0933a658-b1af-4ee1-ab84-f3ea3a12b0ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c663297b4f144f285e9d5eea2ef295e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:54.983020 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0933a658-b1af-4ee1-ab84-f3ea3a12b0ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c663297b4f144f285e9d5eea2ef295e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:55.057145 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1423/2854] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:54 2022] DELETE /identity/v3/users/5c663297b4f144f285e9d5eea2ef295e => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:55.503736 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474471c-181b-412e-bea5-5c0b9e984f85 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:55.505311 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-4474471c-181b-412e-bea5-5c0b9e984f85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:55.519874 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474471c-181b-412e-bea5-5c0b9e984f85 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:55.579705 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61716ebe-c28c-4f71-902c-a68eaea245d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:55.580894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61716ebe-c28c-4f71-902c-a68eaea245d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:55.580894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61716ebe-c28c-4f71-902c-a68eaea245d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:55.581178 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61716ebe-c28c-4f71-902c-a68eaea245d2 admin admin] PATH_INFO: `/v3/projects/12ea419a46dc4ac39e2c32c839cd9692` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:55.582356 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61716ebe-c28c-4f71-902c-a68eaea245d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12ea419a46dc4ac39e2c32c839cd9692)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:55.586961 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61716ebe-c28c-4f71-902c-a68eaea245d2 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:55.711329 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-61716ebe-c28c-4f71-902c-a68eaea245d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12ea419a46dc4ac39e2c32c839cd9692', 'request_id': 'req-61716ebe-c28c-4f71-902c-a68eaea245d2'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:55.719159 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-61716ebe-c28c-4f71-902c-a68eaea245d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12ea419a46dc4ac39e2c32c839cd9692', 'request_id': 'req-61716ebe-c28c-4f71-902c-a68eaea245d2'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:55.743127 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933a658-b1af-4ee1-ab84-f3ea3a12b0ad admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:55.744581 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-0933a658-b1af-4ee1-ab84-f3ea3a12b0ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:55.800452 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-61716ebe-c28c-4f71-902c-a68eaea245d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12ea419a46dc4ac39e2c32c839cd9692', 'request_id': 'req-61716ebe-c28c-4f71-902c-a68eaea245d2'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:55.828064 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-61716ebe-c28c-4f71-902c-a68eaea245d2 admin admin] The token cache is being invalidate because project 12ea419a46dc4ac39e2c32c839cd9692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:55.828064 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-61716ebe-c28c-4f71-902c-a68eaea245d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:55.828064 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-61716ebe-c28c-4f71-902c-a68eaea245d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:55.830556 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1432/2855] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:55 2022] DELETE /identity/v3/projects/12ea419a46dc4ac39e2c32c839cd9692 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:55.842409 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61716ebe-c28c-4f71-902c-a68eaea245d2 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:55.843672 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-61716ebe-c28c-4f71-902c-a68eaea245d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:55.887651 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933a658-b1af-4ee1-ab84-f3ea3a12b0ad admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:55.894433 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:55.928206 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61716ebe-c28c-4f71-902c-a68eaea245d2 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:55.936154 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d553bf2-67b3-4148-b83a-f1b4c255e8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:55.937043 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d553bf2-67b3-4148-b83a-f1b4c255e8eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:55.937326 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d553bf2-67b3-4148-b83a-f1b4c255e8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:55.937603 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d553bf2-67b3-4148-b83a-f1b4c255e8eb admin admin] PATH_INFO: `/v3/users/ae44a59ec0724297bdc3c53597b05b3b` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:55.938842 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d553bf2-67b3-4148-b83a-f1b4c255e8eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae44a59ec0724297bdc3c53597b05b3b)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:55.940786 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d553bf2-67b3-4148-b83a-f1b4c255e8eb admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:55.946463 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:55.967525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f0b11-51b3-4dff-9c84-5c98e02aaa54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac04b32777ca4f929e9e59deee6e0260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:55.968252 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f0b11-51b3-4dff-9c84-5c98e02aaa54 service placement] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:55.968699 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f0b11-51b3-4dff-9c84-5c98e02aaa54 service placement] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:55.968980 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f0b11-51b3-4dff-9c84-5c98e02aaa54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:55.973640 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1d6f0b11-51b3-4dff-9c84-5c98e02aaa54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:55.986377 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f0b11-51b3-4dff-9c84-5c98e02aaa54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:55.988109 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f0b11-51b3-4dff-9c84-5c98e02aaa54 service placement] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:56.002310 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1424/2856] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:57:56.136126 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-0d553bf2-67b3-4148-b83a-f1b4c255e8eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae44a59ec0724297bdc3c53597b05b3b'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:56.143087 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-0d553bf2-67b3-4148-b83a-f1b4c255e8eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae44a59ec0724297bdc3c53597b05b3b'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:56.143880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-0d553bf2-67b3-4148-b83a-f1b4c255e8eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae44a59ec0724297bdc3c53597b05b3b'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:56.145263 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-0d553bf2-67b3-4148-b83a-f1b4c255e8eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae44a59ec0724297bdc3c53597b05b3b'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:56.159130 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-0d553bf2-67b3-4148-b83a-f1b4c255e8eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae44a59ec0724297bdc3c53597b05b3b'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:56.259052 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1433/2857] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:55 2022] DELETE /identity/v3/users/ae44a59ec0724297bdc3c53597b05b3b => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:56.723278 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f0b11-51b3-4dff-9c84-5c98e02aaa54 service placement] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:56.736517 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f0b11-51b3-4dff-9c84-5c98e02aaa54 service placement] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:56.803861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0645dc5-c3ee-46a1-8a58-10432e9e4258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:56.804798 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0645dc5-c3ee-46a1-8a58-10432e9e4258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:56.805113 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0645dc5-c3ee-46a1-8a58-10432e9e4258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:56.805390 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0645dc5-c3ee-46a1-8a58-10432e9e4258 admin admin] PATH_INFO: `/v3/projects/d991d7c4d4a34114ba071eda025bc11c` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:56.806702 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0645dc5-c3ee-46a1-8a58-10432e9e4258 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d991d7c4d4a34114ba071eda025bc11c)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:56.808638 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0645dc5-c3ee-46a1-8a58-10432e9e4258 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:56.932728 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c0645dc5-c3ee-46a1-8a58-10432e9e4258 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd991d7c4d4a34114ba071eda025bc11c', 'request_id': 'req-c0645dc5-c3ee-46a1-8a58-10432e9e4258'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:56.939879 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c0645dc5-c3ee-46a1-8a58-10432e9e4258 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd991d7c4d4a34114ba071eda025bc11c', 'request_id': 'req-c0645dc5-c3ee-46a1-8a58-10432e9e4258'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:57.013269 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c0645dc5-c3ee-46a1-8a58-10432e9e4258 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd991d7c4d4a34114ba071eda025bc11c', 'request_id': 'req-c0645dc5-c3ee-46a1-8a58-10432e9e4258'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:57.024801 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c0645dc5-c3ee-46a1-8a58-10432e9e4258 admin admin] The token cache is being invalidate because project d991d7c4d4a34114ba071eda025bc11c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:57.025086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c0645dc5-c3ee-46a1-8a58-10432e9e4258 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:57.025707 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c0645dc5-c3ee-46a1-8a58-10432e9e4258 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:57.027282 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1425/2858] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:56 2022] DELETE /identity/v3/projects/d991d7c4d4a34114ba071eda025bc11c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:57.078260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d4352-6fd6-47f1-ac1d-4ac7c563e5ac None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:57.078808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d4352-6fd6-47f1-ac1d-4ac7c563e5ac None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:57.078932 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d4352-6fd6-47f1-ac1d-4ac7c563e5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:57.098040 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-552d4352-6fd6-47f1-ac1d-4ac7c563e5ac None None] MFA Rules not processed for user `030614ef56f143f8be6fcc0445ec9568`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:57.099383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-552d4352-6fd6-47f1-ac1d-4ac7c563e5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:57.173751 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1434/2859] 10.222.0.13 () {58 vars in 967 bytes} [Thu Jun 23 11:57:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:57.347099 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0645dc5-c3ee-46a1-8a58-10432e9e4258 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:57.348538 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c0645dc5-c3ee-46a1-8a58-10432e9e4258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:57.419694 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0645dc5-c3ee-46a1-8a58-10432e9e4258 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:57.424636 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731f135-0b9a-4a18-8ce9-002d6cc280f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450b1fad96604fb09d0705895c6d3da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:57.425279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731f135-0b9a-4a18-8ce9-002d6cc280f6 service swift] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:57.425560 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731f135-0b9a-4a18-8ce9-002d6cc280f6 service swift] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:57.425837 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731f135-0b9a-4a18-8ce9-002d6cc280f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:57.435940 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731f135-0b9a-4a18-8ce9-002d6cc280f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:57.437512 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731f135-0b9a-4a18-8ce9-002d6cc280f6 service swift] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:57.440992 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:57.446500 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1426/2860] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 23 11:57:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:58:01.962266 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d4352-6fd6-47f1-ac1d-4ac7c563e5ac None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:01.963747 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-552d4352-6fd6-47f1-ac1d-4ac7c563e5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:02.029274 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:58:02.035284 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d4352-6fd6-47f1-ac1d-4ac7c563e5ac None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:02.040326 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdcc84-e73f-4017-bbf2-e747179bf1e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac04b32777ca4f929e9e59deee6e0260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:02.040971 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdcc84-e73f-4017-bbf2-e747179bf1e9 service placement] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:02.041288 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdcc84-e73f-4017-bbf2-e747179bf1e9 service placement] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:02.041565 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdcc84-e73f-4017-bbf2-e747179bf1e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:02.043346 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-60cdcc84-e73f-4017-bbf2-e747179bf1e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:02.054758 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cdcc84-e73f-4017-bbf2-e747179bf1e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:02.056358 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cdcc84-e73f-4017-bbf2-e747179bf1e9 service placement] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:02.066303 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1435/2861] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:58:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:58:04.169659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e714cc-6b53-479d-8855-5a8f7d721efd None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:04.170171 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e714cc-6b53-479d-8855-5a8f7d721efd None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:04.170307 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e714cc-6b53-479d-8855-5a8f7d721efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:04.190013 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b6e714cc-6b53-479d-8855-5a8f7d721efd None None] MFA Rules not processed for user `030614ef56f143f8be6fcc0445ec9568`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:58:04.191275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b6e714cc-6b53-479d-8855-5a8f7d721efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:04.199101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1427/2862] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 23 11:58:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:58:04.213256 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdcc84-e73f-4017-bbf2-e747179bf1e9 service placement] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:04.225590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdcc84-e73f-4017-bbf2-e747179bf1e9 service placement] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:04.231254 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c456a76-3274-4f62-87de-0f6ee0466d56 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450b1fad96604fb09d0705895c6d3da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:04.232109 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c456a76-3274-4f62-87de-0f6ee0466d56 service swift] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:04.232981 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c456a76-3274-4f62-87de-0f6ee0466d56 service swift] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:04.233420 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c456a76-3274-4f62-87de-0f6ee0466d56 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:04.243302 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c456a76-3274-4f62-87de-0f6ee0466d56 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:04.245260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c456a76-3274-4f62-87de-0f6ee0466d56 service swift] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:04.254301 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1436/2863] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 23 11:58:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:58:06.075076 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e714cc-6b53-479d-8855-5a8f7d721efd None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:06.075076 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b6e714cc-6b53-479d-8855-5a8f7d721efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:06.161641 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e714cc-6b53-479d-8855-5a8f7d721efd None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:06.168110 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6f28f-7e01-44a0-be0b-a314f8212efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:06.169057 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6f28f-7e01-44a0-be0b-a314f8212efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:06.169414 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6f28f-7e01-44a0-be0b-a314f8212efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:06.169674 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6f28f-7e01-44a0-be0b-a314f8212efb admin admin] PATH_INFO: `/v3/users/15a4c8e15cd44904805c2519c4b6864a` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:06.170970 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6f28f-7e01-44a0-be0b-a314f8212efb admin admin] RBAC: Authorizing `identity:delete_user(user_id=15a4c8e15cd44904805c2519c4b6864a)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:06.173264 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6f28f-7e01-44a0-be0b-a314f8212efb admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:06.373980 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-35a6f28f-7e01-44a0-be0b-a314f8212efb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15a4c8e15cd44904805c2519c4b6864a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:06.382165 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-35a6f28f-7e01-44a0-be0b-a314f8212efb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15a4c8e15cd44904805c2519c4b6864a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:06.383105 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-35a6f28f-7e01-44a0-be0b-a314f8212efb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15a4c8e15cd44904805c2519c4b6864a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:06.384001 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-35a6f28f-7e01-44a0-be0b-a314f8212efb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15a4c8e15cd44904805c2519c4b6864a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:06.399710 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-35a6f28f-7e01-44a0-be0b-a314f8212efb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15a4c8e15cd44904805c2519c4b6864a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:06.455166 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1428/2864] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:58:06 2022] DELETE /identity/v3/users/15a4c8e15cd44904805c2519c4b6864a => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:58:06.991398 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c456a76-3274-4f62-87de-0f6ee0466d56 service swift] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:07.010330 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c456a76-3274-4f62-87de-0f6ee0466d56 service swift] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:07.123637 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a6148-0f28-48ae-b753-cc2e0b9aceb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:07.125107 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a6148-0f28-48ae-b753-cc2e0b9aceb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:07.125631 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a6148-0f28-48ae-b753-cc2e0b9aceb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:07.126135 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a6148-0f28-48ae-b753-cc2e0b9aceb5 admin admin] PATH_INFO: `/v3/projects/17a1542e108d4e3e85fe5525a7e331d0` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:07.127992 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a6148-0f28-48ae-b753-cc2e0b9aceb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17a1542e108d4e3e85fe5525a7e331d0)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:07.131747 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a6148-0f28-48ae-b753-cc2e0b9aceb5 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:07.257615 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6a9a6148-0f28-48ae-b753-cc2e0b9aceb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17a1542e108d4e3e85fe5525a7e331d0', 'request_id': 'req-6a9a6148-0f28-48ae-b753-cc2e0b9aceb5'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:07.264764 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6a9a6148-0f28-48ae-b753-cc2e0b9aceb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17a1542e108d4e3e85fe5525a7e331d0', 'request_id': 'req-6a9a6148-0f28-48ae-b753-cc2e0b9aceb5'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:07.424688 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6a9a6148-0f28-48ae-b753-cc2e0b9aceb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17a1542e108d4e3e85fe5525a7e331d0', 'request_id': 'req-6a9a6148-0f28-48ae-b753-cc2e0b9aceb5'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:07.439297 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6a9a6148-0f28-48ae-b753-cc2e0b9aceb5 admin admin] The token cache is being invalidate because project 17a1542e108d4e3e85fe5525a7e331d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:58:07.439639 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6a9a6148-0f28-48ae-b753-cc2e0b9aceb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:07.440483 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-6a9a6148-0f28-48ae-b753-cc2e0b9aceb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:07.443937 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1437/2865] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:58:06 2022] DELETE /identity/v3/projects/17a1542e108d4e3e85fe5525a7e331d0 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:58:07.448571 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6f28f-7e01-44a0-be0b-a314f8212efb admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:07.450095 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-35a6f28f-7e01-44a0-be0b-a314f8212efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:07.465582 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6f28f-7e01-44a0-be0b-a314f8212efb admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:07.543570 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cfa37-52ce-466f-b1da-f0dd4fc8d9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:07.544719 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cfa37-52ce-466f-b1da-f0dd4fc8d9b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:07.545083 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cfa37-52ce-466f-b1da-f0dd4fc8d9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:07.545416 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cfa37-52ce-466f-b1da-f0dd4fc8d9b2 admin admin] PATH_INFO: `/v3/users/dbfc00a8fc9f4b1f9535619c5a957c81` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:07.546927 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0cfa37-52ce-466f-b1da-f0dd4fc8d9b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbfc00a8fc9f4b1f9535619c5a957c81)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:07.549336 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0cfa37-52ce-466f-b1da-f0dd4fc8d9b2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:07.565253 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:58:07.819800 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5e0cfa37-52ce-466f-b1da-f0dd4fc8d9b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbfc00a8fc9f4b1f9535619c5a957c81'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:07.827799 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5e0cfa37-52ce-466f-b1da-f0dd4fc8d9b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbfc00a8fc9f4b1f9535619c5a957c81'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:07.828505 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5e0cfa37-52ce-466f-b1da-f0dd4fc8d9b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbfc00a8fc9f4b1f9535619c5a957c81'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:07.829260 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5e0cfa37-52ce-466f-b1da-f0dd4fc8d9b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbfc00a8fc9f4b1f9535619c5a957c81'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:07.846458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5e0cfa37-52ce-466f-b1da-f0dd4fc8d9b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbfc00a8fc9f4b1f9535619c5a957c81'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:07.922778 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1429/2866] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:58:07 2022] DELETE /identity/v3/users/dbfc00a8fc9f4b1f9535619c5a957c81 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:58:08.271419 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a6148-0f28-48ae-b753-cc2e0b9aceb5 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:08.274693 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-6a9a6148-0f28-48ae-b753-cc2e0b9aceb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:08.319513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a6148-0f28-48ae-b753-cc2e0b9aceb5 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:08.391801 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32aac0-c2be-4933-a311-79713bb5a091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:08.393061 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32aac0-c2be-4933-a311-79713bb5a091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:08.393410 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32aac0-c2be-4933-a311-79713bb5a091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:08.393747 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32aac0-c2be-4933-a311-79713bb5a091 admin admin] PATH_INFO: `/v3/projects/5fdf048be460453d937439116caa2f4f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:08.395161 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c32aac0-c2be-4933-a311-79713bb5a091 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fdf048be460453d937439116caa2f4f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:08.397959 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c32aac0-c2be-4933-a311-79713bb5a091 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:08.407308 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:58:08.524906 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-1c32aac0-c2be-4933-a311-79713bb5a091 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fdf048be460453d937439116caa2f4f', 'request_id': 'req-1c32aac0-c2be-4933-a311-79713bb5a091'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:08.532465 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-1c32aac0-c2be-4933-a311-79713bb5a091 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fdf048be460453d937439116caa2f4f', 'request_id': 'req-1c32aac0-c2be-4933-a311-79713bb5a091'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:08.591128 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-1c32aac0-c2be-4933-a311-79713bb5a091 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fdf048be460453d937439116caa2f4f', 'request_id': 'req-1c32aac0-c2be-4933-a311-79713bb5a091'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:08.607009 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-1c32aac0-c2be-4933-a311-79713bb5a091 admin admin] The token cache is being invalidate because project 5fdf048be460453d937439116caa2f4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:58:08.607698 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-1c32aac0-c2be-4933-a311-79713bb5a091 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:08.608407 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-1c32aac0-c2be-4933-a311-79713bb5a091 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:08.618582 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1438/2867] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:58:08 2022] DELETE /identity/v3/projects/5fdf048be460453d937439116caa2f4f => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:58:17.915579 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=22,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:58:18.603362 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:58:20.891640 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb61e6-ed93-4dfe-be15-35a21d7e6518 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:20.892143 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb61e6-ed93-4dfe-be15-35a21d7e6518 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:20.892279 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb61e6-ed93-4dfe-be15-35a21d7e6518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:20.912383 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-adfb61e6-ed93-4dfe-be15-35a21d7e6518 None None] MFA Rules not processed for user `030614ef56f143f8be6fcc0445ec9568`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:58:20.913678 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-adfb61e6-ed93-4dfe-be15-35a21d7e6518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:20.968507 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1430/2868] 10.222.0.13 () {58 vars in 967 bytes} [Thu Jun 23 11:58:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:58:20.980744 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32aac0-c2be-4933-a311-79713bb5a091 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:20.982122 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-1c32aac0-c2be-4933-a311-79713bb5a091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:21.051144 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32aac0-c2be-4933-a311-79713bb5a091 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:21.056689 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902e80e-34ac-4702-addd-a58a3f39eb82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450b1fad96604fb09d0705895c6d3da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:21.057341 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902e80e-34ac-4702-addd-a58a3f39eb82 service swift] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:21.057627 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902e80e-34ac-4702-addd-a58a3f39eb82 service swift] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:21.057905 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902e80e-34ac-4702-addd-a58a3f39eb82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:21.065730 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902e80e-34ac-4702-addd-a58a3f39eb82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:21.067358 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902e80e-34ac-4702-addd-a58a3f39eb82 service swift] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:21.075763 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1439/2869] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 23 11:58:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:58:24.026364 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb61e6-ed93-4dfe-be15-35a21d7e6518 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:24.027822 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-adfb61e6-ed93-4dfe-be15-35a21d7e6518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:24.099274 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb61e6-ed93-4dfe-be15-35a21d7e6518 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:24.104070 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849035b-527b-458a-99eb-f88e2b16e176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:24.104937 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849035b-527b-458a-99eb-f88e2b16e176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:24.105216 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849035b-527b-458a-99eb-f88e2b16e176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:24.105599 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849035b-527b-458a-99eb-f88e2b16e176 admin admin] PATH_INFO: `/v3/users/69c345b278e4437d98f9b1f412bbca49` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:24.106729 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849035b-527b-458a-99eb-f88e2b16e176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69c345b278e4437d98f9b1f412bbca49)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:24.108914 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849035b-527b-458a-99eb-f88e2b16e176 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:24.311274 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6849035b-527b-458a-99eb-f88e2b16e176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69c345b278e4437d98f9b1f412bbca49'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:24.318233 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6849035b-527b-458a-99eb-f88e2b16e176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69c345b278e4437d98f9b1f412bbca49'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:24.318958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6849035b-527b-458a-99eb-f88e2b16e176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69c345b278e4437d98f9b1f412bbca49'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:24.319560 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6849035b-527b-458a-99eb-f88e2b16e176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69c345b278e4437d98f9b1f412bbca49'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:24.331933 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6849035b-527b-458a-99eb-f88e2b16e176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69c345b278e4437d98f9b1f412bbca49'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:24.420373 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1431/2870] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:58:24 2022] DELETE /identity/v3/users/69c345b278e4437d98f9b1f412bbca49 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:58:24.677211 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902e80e-34ac-4702-addd-a58a3f39eb82 service swift] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:24.688683 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902e80e-34ac-4702-addd-a58a3f39eb82 service swift] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:24.746429 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d839c-65f2-438c-a46b-8f2f839421df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:24.747319 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d839c-65f2-438c-a46b-8f2f839421df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:24.747612 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d839c-65f2-438c-a46b-8f2f839421df admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:24.747894 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d839c-65f2-438c-a46b-8f2f839421df admin admin] PATH_INFO: `/v3/projects/a00e3dc3a73a42bdb9426b0213933056` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:24.749072 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d839c-65f2-438c-a46b-8f2f839421df admin admin] RBAC: Authorizing `identity:delete_project(project_id=a00e3dc3a73a42bdb9426b0213933056)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:24.751012 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d839c-65f2-438c-a46b-8f2f839421df admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:24.898203 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-470d839c-65f2-438c-a46b-8f2f839421df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a00e3dc3a73a42bdb9426b0213933056', 'request_id': 'req-470d839c-65f2-438c-a46b-8f2f839421df'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:24.904370 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-470d839c-65f2-438c-a46b-8f2f839421df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a00e3dc3a73a42bdb9426b0213933056', 'request_id': 'req-470d839c-65f2-438c-a46b-8f2f839421df'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:25.011281 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-470d839c-65f2-438c-a46b-8f2f839421df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a00e3dc3a73a42bdb9426b0213933056', 'request_id': 'req-470d839c-65f2-438c-a46b-8f2f839421df'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:25.021904 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-470d839c-65f2-438c-a46b-8f2f839421df admin admin] The token cache is being invalidate because project a00e3dc3a73a42bdb9426b0213933056 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:58:25.022194 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-470d839c-65f2-438c-a46b-8f2f839421df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:25.022891 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-470d839c-65f2-438c-a46b-8f2f839421df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:25.024547 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1440/2871] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:58:24 2022] DELETE /identity/v3/projects/a00e3dc3a73a42bdb9426b0213933056 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:58:25.035642 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c9379-450b-4c7b-8c0b-bfb1c448fc1e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:25.036150 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c9379-450b-4c7b-8c0b-bfb1c448fc1e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:25.036262 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c9379-450b-4c7b-8c0b-bfb1c448fc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:25.055607 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-760c9379-450b-4c7b-8c0b-bfb1c448fc1e None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:58:25.056841 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-760c9379-450b-4c7b-8c0b-bfb1c448fc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:25.074951 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:58:25.114705 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1432/2872] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:58:25.123360 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafadfcd-dcd3-4969-bfe8-1ec95f1c2536 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:25.123774 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafadfcd-dcd3-4969-bfe8-1ec95f1c2536 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:25.124166 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafadfcd-dcd3-4969-bfe8-1ec95f1c2536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:25.143419 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-cafadfcd-dcd3-4969-bfe8-1ec95f1c2536 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:58:25.144591 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-cafadfcd-dcd3-4969-bfe8-1ec95f1c2536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:25.152795 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1441/2873] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:58:25.162241 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c9379-450b-4c7b-8c0b-bfb1c448fc1e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:25.167199 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c9379-450b-4c7b-8c0b-bfb1c448fc1e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:25.171738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a8a6c-1201-486e-aed6-a8038dfb870f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:25.172381 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a8a6c-1201-486e-aed6-a8038dfb870f admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:25.172663 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a8a6c-1201-486e-aed6-a8038dfb870f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:25.172935 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a8a6c-1201-486e-aed6-a8038dfb870f admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:25.173657 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a8a6c-1201-486e-aed6-a8038dfb870f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:58:25.173961 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a8a6c-1201-486e-aed6-a8038dfb870f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:25.175611 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a8a6c-1201-486e-aed6-a8038dfb870f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:25.187895 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1433/2874] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:58:25.192121 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafadfcd-dcd3-4969-bfe8-1ec95f1c2536 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:25.201572 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafadfcd-dcd3-4969-bfe8-1ec95f1c2536 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:25.206273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ada99-a0b5-4aea-a52d-6c8a3868c62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:25.206971 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ada99-a0b5-4aea-a52d-6c8a3868c62d admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:25.207234 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ada99-a0b5-4aea-a52d-6c8a3868c62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:25.207508 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ada99-a0b5-4aea-a52d-6c8a3868c62d admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:25.208899 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ada99-a0b5-4aea-a52d-6c8a3868c62d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:25.210936 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ada99-a0b5-4aea-a52d-6c8a3868c62d admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:25.220947 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:58:25.282972 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1442/2875] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:58:25 2022] POST /identity/v3/projects => generated 369 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:58:25.287013 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a8a6c-1201-486e-aed6-a8038dfb870f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:25.297480 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a8a6c-1201-486e-aed6-a8038dfb870f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:25.352810 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e292b-e82c-4928-be3a-dd50dc881003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:25.353456 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e292b-e82c-4928-be3a-dd50dc881003 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:25.353733 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e292b-e82c-4928-be3a-dd50dc881003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:25.354003 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e292b-e82c-4928-be3a-dd50dc881003 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:25.354708 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e292b-e82c-4928-be3a-dd50dc881003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:25.356630 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e292b-e82c-4928-be3a-dd50dc881003 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:25.447585 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1434/2876] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:58:25 2022] POST /identity/v3/users => generated 285 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:58:25.451723 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ada99-a0b5-4aea-a52d-6c8a3868c62d admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:25.462454 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ada99-a0b5-4aea-a52d-6c8a3868c62d admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:25.467085 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f521373-6773-4cd5-bfbc-7821a86d6f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:25.467735 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f521373-6773-4cd5-bfbc-7821a86d6f04 admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:25.468016 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f521373-6773-4cd5-bfbc-7821a86d6f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:25.468484 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f521373-6773-4cd5-bfbc-7821a86d6f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:25.469003 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f521373-6773-4cd5-bfbc-7821a86d6f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:25.470616 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f521373-6773-4cd5-bfbc-7821a86d6f04 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:25.481599 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1443/2877] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:58:25.485810 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e292b-e82c-4928-be3a-dd50dc881003 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:25.499112 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e292b-e82c-4928-be3a-dd50dc881003 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:25.503983 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61458ae8-3f0b-4ba0-8c2c-5594deddb2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:25.504639 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61458ae8-3f0b-4ba0-8c2c-5594deddb2e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:25.504918 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61458ae8-3f0b-4ba0-8c2c-5594deddb2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:25.505190 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61458ae8-3f0b-4ba0-8c2c-5594deddb2e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:25.505902 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61458ae8-3f0b-4ba0-8c2c-5594deddb2e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:25.507530 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61458ae8-3f0b-4ba0-8c2c-5594deddb2e7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:25.517861 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1435/2878] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:58:25.522274 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f521373-6773-4cd5-bfbc-7821a86d6f04 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:25.533191 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f521373-6773-4cd5-bfbc-7821a86d6f04 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:25.537729 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b363ba2-0a37-4c8a-8ff2-e2af745a8f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:25.539278 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b363ba2-0a37-4c8a-8ff2-e2af745a8f57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:25.539563 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b363ba2-0a37-4c8a-8ff2-e2af745a8f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:25.539842 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b363ba2-0a37-4c8a-8ff2-e2af745a8f57 admin admin] PATH_INFO: `/v3/projects/c6f36366fd4c4123acb1e5890c6af356/users/a145b4a2a30d4fb9b0393f94b88dae03/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:25.554854 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b363ba2-0a37-4c8a-8ff2-e2af745a8f57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6f36366fd4c4123acb1e5890c6af356, user_id=a145b4a2a30d4fb9b0393f94b88dae03, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:25.557590 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b363ba2-0a37-4c8a-8ff2-e2af745a8f57 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:25.641596 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1444/2879] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:58:25 2022] PUT /identity/v3/projects/c6f36366fd4c4123acb1e5890c6af356/users/a145b4a2a30d4fb9b0393f94b88dae03/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:58:25.653135 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3ecee-1742-41d8-824f-b7171340780d None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:25.653465 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3ecee-1742-41d8-824f-b7171340780d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:25.653738 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3ecee-1742-41d8-824f-b7171340780d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:25.680497 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-15b3ecee-1742-41d8-824f-b7171340780d None None] MFA Rules not processed for user `a145b4a2a30d4fb9b0393f94b88dae03`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:58:25.681698 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-15b3ecee-1742-41d8-824f-b7171340780d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:25.745735 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1436/2880] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:58:25 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:58:25.758067 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b363ba2-0a37-4c8a-8ff2-e2af745a8f57 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:25.763305 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-4b363ba2-0a37-4c8a-8ff2-e2af745a8f57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:25.815736 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b363ba2-0a37-4c8a-8ff2-e2af745a8f57 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:25.819492 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c91688-121f-42f4-ad1c-bfee6748403e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6105838cbf1409587ee7d9e3be554f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:25.820679 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c91688-121f-42f4-ad1c-bfee6748403e service glance] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:25.821048 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c91688-121f-42f4-ad1c-bfee6748403e service glance] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:25.821325 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c91688-121f-42f4-ad1c-bfee6748403e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:25.829685 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c91688-121f-42f4-ad1c-bfee6748403e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:25.831159 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c91688-121f-42f4-ad1c-bfee6748403e service glance] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:25.840458 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1445/2881] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:58:25 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:58:35.742808 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:58:35.835788 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:58:59.397213 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3ecee-1742-41d8-824f-b7171340780d None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:59.459001 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3ecee-1742-41d8-824f-b7171340780d None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:59.463581 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205691de-8bde-4b58-b5a5-b75ef1201780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:59.464445 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205691de-8bde-4b58-b5a5-b75ef1201780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:59.464730 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205691de-8bde-4b58-b5a5-b75ef1201780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:59.465005 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205691de-8bde-4b58-b5a5-b75ef1201780 admin admin] PATH_INFO: `/v3/users/a145b4a2a30d4fb9b0393f94b88dae03` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:59.466189 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205691de-8bde-4b58-b5a5-b75ef1201780 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a145b4a2a30d4fb9b0393f94b88dae03)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:59.468242 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205691de-8bde-4b58-b5a5-b75ef1201780 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:59.687085 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-205691de-8bde-4b58-b5a5-b75ef1201780 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a145b4a2a30d4fb9b0393f94b88dae03'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:59.695175 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-205691de-8bde-4b58-b5a5-b75ef1201780 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a145b4a2a30d4fb9b0393f94b88dae03'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:59.695847 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-205691de-8bde-4b58-b5a5-b75ef1201780 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a145b4a2a30d4fb9b0393f94b88dae03'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:59.696460 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-205691de-8bde-4b58-b5a5-b75ef1201780 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a145b4a2a30d4fb9b0393f94b88dae03'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:59.708531 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-205691de-8bde-4b58-b5a5-b75ef1201780 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a145b4a2a30d4fb9b0393f94b88dae03'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:59.790026 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1437/2882] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:58:59 2022] DELETE /identity/v3/users/a145b4a2a30d4fb9b0393f94b88dae03 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:58:59.794523 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f3501-d367-48f6-99be-75adcf237329 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:59.795139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f3501-d367-48f6-99be-75adcf237329 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:59.795139 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f3501-d367-48f6-99be-75adcf237329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:59.812963 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-4b5f3501-d367-48f6-99be-75adcf237329 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:58:59.815017 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-4b5f3501-d367-48f6-99be-75adcf237329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:59.872272 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1446/2883] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:58:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:58:59.882476 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205691de-8bde-4b58-b5a5-b75ef1201780 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:59.883891 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-205691de-8bde-4b58-b5a5-b75ef1201780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:59.941806 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205691de-8bde-4b58-b5a5-b75ef1201780 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:59.945326 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a644c17-7ec3-4f2a-8cb9-5809a598405c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:59.945968 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a644c17-7ec3-4f2a-8cb9-5809a598405c service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:59.946251 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a644c17-7ec3-4f2a-8cb9-5809a598405c service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:59.946525 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a644c17-7ec3-4f2a-8cb9-5809a598405c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:59.955508 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a644c17-7ec3-4f2a-8cb9-5809a598405c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:59.956979 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a644c17-7ec3-4f2a-8cb9-5809a598405c service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:59.966620 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1438/2884] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:58:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:59:00.360880 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f3501-d367-48f6-99be-75adcf237329 None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:00.362825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-4b5f3501-d367-48f6-99be-75adcf237329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:00.374617 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f3501-d367-48f6-99be-75adcf237329 None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:00.379759 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44273a7-a3f3-49ae-8c4e-9c58b2595663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:00.380634 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44273a7-a3f3-49ae-8c4e-9c58b2595663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:00.380916 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44273a7-a3f3-49ae-8c4e-9c58b2595663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:00.381192 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44273a7-a3f3-49ae-8c4e-9c58b2595663 admin admin] PATH_INFO: `/v3/projects/c6f36366fd4c4123acb1e5890c6af356` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:00.382393 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44273a7-a3f3-49ae-8c4e-9c58b2595663 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6f36366fd4c4123acb1e5890c6af356)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:00.384380 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44273a7-a3f3-49ae-8c4e-9c58b2595663 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:00.503576 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-e44273a7-a3f3-49ae-8c4e-9c58b2595663 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6f36366fd4c4123acb1e5890c6af356', 'request_id': 'req-e44273a7-a3f3-49ae-8c4e-9c58b2595663'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:00.509733 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-e44273a7-a3f3-49ae-8c4e-9c58b2595663 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6f36366fd4c4123acb1e5890c6af356', 'request_id': 'req-e44273a7-a3f3-49ae-8c4e-9c58b2595663'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:00.578383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-e44273a7-a3f3-49ae-8c4e-9c58b2595663 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6f36366fd4c4123acb1e5890c6af356', 'request_id': 'req-e44273a7-a3f3-49ae-8c4e-9c58b2595663'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:00.589353 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-e44273a7-a3f3-49ae-8c4e-9c58b2595663 admin admin] The token cache is being invalidate because project c6f36366fd4c4123acb1e5890c6af356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80255) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:59:00.589647 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-e44273a7-a3f3-49ae-8c4e-9c58b2595663 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:00.590264 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-e44273a7-a3f3-49ae-8c4e-9c58b2595663 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:00.591816 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1447/2885] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:59:00 2022] DELETE /identity/v3/projects/c6f36366fd4c4123acb1e5890c6af356 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:59:00.597485 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2870d46-f626-44c4-a204-943ac632985b None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:00.597971 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2870d46-f626-44c4-a204-943ac632985b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:00.598078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2870d46-f626-44c4-a204-943ac632985b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:00.615427 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c2870d46-f626-44c4-a204-943ac632985b None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:00.616618 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c2870d46-f626-44c4-a204-943ac632985b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:00.633054 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:00.677409 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1439/2886] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:59:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:00.686261 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1afcf-3ecb-46e5-8b1b-ee5507bafb0d None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:00.686580 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1afcf-3ecb-46e5-8b1b-ee5507bafb0d None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:00.686866 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1afcf-3ecb-46e5-8b1b-ee5507bafb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:00.706115 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-74d1afcf-3ecb-46e5-8b1b-ee5507bafb0d None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:00.707344 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-74d1afcf-3ecb-46e5-8b1b-ee5507bafb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:00.715185 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1448/2887] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:59:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:00.719433 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2870d46-f626-44c4-a204-943ac632985b None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:00.730011 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2870d46-f626-44c4-a204-943ac632985b None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:00.735087 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cd667-7351-4e59-ab6b-872792070534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:00.735737 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cd667-7351-4e59-ab6b-872792070534 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:00.736023 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cd667-7351-4e59-ab6b-872792070534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:00.736419 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cd667-7351-4e59-ab6b-872792070534 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:00.737182 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46cd667-7351-4e59-ab6b-872792070534 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:59:00.737501 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46cd667-7351-4e59-ab6b-872792070534 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:00.739220 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46cd667-7351-4e59-ab6b-872792070534 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:00.751101 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1440/2888] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 23 11:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:59:00.755674 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1afcf-3ecb-46e5-8b1b-ee5507bafb0d None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:00.765004 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1afcf-3ecb-46e5-8b1b-ee5507bafb0d None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:00.770294 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5be77e-9c88-42a6-ac3f-550770abb8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:00.770976 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5be77e-9c88-42a6-ac3f-550770abb8c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:00.771262 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5be77e-9c88-42a6-ac3f-550770abb8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:00.771566 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5be77e-9c88-42a6-ac3f-550770abb8c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:00.772260 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5be77e-9c88-42a6-ac3f-550770abb8c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:00.774158 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5be77e-9c88-42a6-ac3f-550770abb8c8 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:00.871248 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1449/2889] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 23 11:59:00 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:59:00.875518 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cd667-7351-4e59-ab6b-872792070534 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:00.886394 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cd667-7351-4e59-ab6b-872792070534 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:00.942772 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c68a2-3cfc-49ab-bb37-e504764af238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:00.943485 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c68a2-3cfc-49ab-bb37-e504764af238 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:00.943806 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c68a2-3cfc-49ab-bb37-e504764af238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:00.944086 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c68a2-3cfc-49ab-bb37-e504764af238 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:00.944785 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c68a2-3cfc-49ab-bb37-e504764af238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:00.946781 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c68a2-3cfc-49ab-bb37-e504764af238 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:01.027427 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1441/2890] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 23 11:59:00 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:59:01.031513 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5be77e-9c88-42a6-ac3f-550770abb8c8 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:01.041813 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5be77e-9c88-42a6-ac3f-550770abb8c8 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:01.046356 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c461e-d147-4ce6-84c3-4a79cf8c275e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:01.053486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c461e-d147-4ce6-84c3-4a79cf8c275e admin admin] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:01.053486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c461e-d147-4ce6-84c3-4a79cf8c275e admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:01.053486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c461e-d147-4ce6-84c3-4a79cf8c275e admin admin] PATH_INFO: `/v3/roles` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:01.053486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c461e-d147-4ce6-84c3-4a79cf8c275e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:01.053486 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c461e-d147-4ce6-84c3-4a79cf8c275e admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:01.060978 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1450/2891] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:59:01.065526 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c68a2-3cfc-49ab-bb37-e504764af238 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:01.075476 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c68a2-3cfc-49ab-bb37-e504764af238 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:01.080128 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e621ac9-134c-44a0-9c80-5f2d4bd48e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:01.080787 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e621ac9-134c-44a0-9c80-5f2d4bd48e32 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:01.081068 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e621ac9-134c-44a0-9c80-5f2d4bd48e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:01.081344 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e621ac9-134c-44a0-9c80-5f2d4bd48e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:01.082067 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e621ac9-134c-44a0-9c80-5f2d4bd48e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:01.083679 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e621ac9-134c-44a0-9c80-5f2d4bd48e32 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:01.093640 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1442/2892] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 23 11:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:59:01.097870 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c461e-d147-4ce6-84c3-4a79cf8c275e admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:01.107600 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c461e-d147-4ce6-84c3-4a79cf8c275e admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:01.113369 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db112242-eb9a-4060-a10e-5d5f769e8b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:01.114233 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db112242-eb9a-4060-a10e-5d5f769e8b31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:01.114521 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db112242-eb9a-4060-a10e-5d5f769e8b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:01.114825 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db112242-eb9a-4060-a10e-5d5f769e8b31 admin admin] PATH_INFO: `/v3/projects/f3934169b16f4e589b7670d234a38252/users/9153568c8f0b4037afabf05f23da3459/roles/75f08cd0671f436c9f968976aa8baa7f` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:01.129576 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db112242-eb9a-4060-a10e-5d5f769e8b31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3934169b16f4e589b7670d234a38252, user_id=9153568c8f0b4037afabf05f23da3459, role_id=75f08cd0671f436c9f968976aa8baa7f)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:01.132281 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db112242-eb9a-4060-a10e-5d5f769e8b31 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:01.212855 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1451/2893] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 23 11:59:01 2022] PUT /identity/v3/projects/f3934169b16f4e589b7670d234a38252/users/9153568c8f0b4037afabf05f23da3459/roles/75f08cd0671f436c9f968976aa8baa7f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:59:01.235558 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ed0d7-3e24-47f1-be0d-9830f40a8101 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:01.235882 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ed0d7-3e24-47f1-be0d-9830f40a8101 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:01.236667 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ed0d7-3e24-47f1-be0d-9830f40a8101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:01.263661 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-956ed0d7-3e24-47f1-be0d-9830f40a8101 None None] MFA Rules not processed for user `9153568c8f0b4037afabf05f23da3459`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:01.264856 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-956ed0d7-3e24-47f1-be0d-9830f40a8101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:01.327701 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1443/2894] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:59:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:01.338454 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db112242-eb9a-4060-a10e-5d5f769e8b31 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:01.340218 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-db112242-eb9a-4060-a10e-5d5f769e8b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:01.362995 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:01.397119 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db112242-eb9a-4060-a10e-5d5f769e8b31 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:01.400750 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce8515-2fb4-4971-8d36-85a51d70ba3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '871dd684032c49d09608f0af05b7a765', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:01.401387 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce8515-2fb4-4971-8d36-85a51d70ba3a service cinder] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:01.401672 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce8515-2fb4-4971-8d36-85a51d70ba3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:01.401950 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce8515-2fb4-4971-8d36-85a51d70ba3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:01.410931 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce8515-2fb4-4971-8d36-85a51d70ba3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:01.412361 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce8515-2fb4-4971-8d36-85a51d70ba3a service cinder] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:01.421024 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1452/2895] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:59:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:59:03.174015 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea1a9d-7da9-410c-a013-1e7f3a651c56 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:03.174633 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea1a9d-7da9-410c-a013-1e7f3a651c56 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:03.174633 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea1a9d-7da9-410c-a013-1e7f3a651c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:03.193020 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-94ea1a9d-7da9-410c-a013-1e7f3a651c56 None None] MFA Rules not processed for user `9153568c8f0b4037afabf05f23da3459`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:03.194212 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-94ea1a9d-7da9-410c-a013-1e7f3a651c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:03.201036 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1444/2896] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:59:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:03.236698 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce8515-2fb4-4971-8d36-85a51d70ba3a service cinder] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:03.238846 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-88ce8515-2fb4-4971-8d36-85a51d70ba3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:03.296479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce8515-2fb4-4971-8d36-85a51d70ba3a service cinder] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:03.300241 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de989f62-8812-4f08-8668-17598ce076ad service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6105838cbf1409587ee7d9e3be554f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:03.307695 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de989f62-8812-4f08-8668-17598ce076ad service glance] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:03.307695 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de989f62-8812-4f08-8668-17598ce076ad service glance] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:03.307695 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de989f62-8812-4f08-8668-17598ce076ad service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:03.307695 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-de989f62-8812-4f08-8668-17598ce076ad service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:03.375273 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de989f62-8812-4f08-8668-17598ce076ad service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:03.376734 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de989f62-8812-4f08-8668-17598ce076ad service glance] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:03.387707 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1453/2897] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:59:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:59:06.107746 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea1a9d-7da9-410c-a013-1e7f3a651c56 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:06.109505 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-94ea1a9d-7da9-410c-a013-1e7f3a651c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:06.124132 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:06.169779 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea1a9d-7da9-410c-a013-1e7f3a651c56 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:06.175891 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead613f-d18a-4b50-abd3-84933ce22daf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:06.177505 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead613f-d18a-4b50-abd3-84933ce22daf service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:06.177795 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead613f-d18a-4b50-abd3-84933ce22daf service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:06.178084 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead613f-d18a-4b50-abd3-84933ce22daf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:06.179874 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7ead613f-d18a-4b50-abd3-84933ce22daf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:06.194514 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead613f-d18a-4b50-abd3-84933ce22daf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:06.196078 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead613f-d18a-4b50-abd3-84933ce22daf service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:06.211196 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1445/2898] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:59:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:59:13.382924 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=71 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:16.320089 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=21 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:22.210116 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece6012-e97f-4902-ab45-0ce06d8fa87b None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:22.211105 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece6012-e97f-4902-ab45-0ce06d8fa87b None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:22.211292 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece6012-e97f-4902-ab45-0ce06d8fa87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:22.349918 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-9ece6012-e97f-4902-ab45-0ce06d8fa87b None None] MFA Rules not processed for user `030614ef56f143f8be6fcc0445ec9568`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:22.351085 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-9ece6012-e97f-4902-ab45-0ce06d8fa87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:22.405129 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1454/2899] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 23 11:59:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:22.548794 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead613f-d18a-4b50-abd3-84933ce22daf service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:22.550284 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7ead613f-d18a-4b50-abd3-84933ce22daf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:22.619293 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead613f-d18a-4b50-abd3-84933ce22daf service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:22.624159 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb83f28-6bcf-45d1-9c13-ed753e2c728e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450b1fad96604fb09d0705895c6d3da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:22.624788 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb83f28-6bcf-45d1-9c13-ed753e2c728e service swift] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:22.625066 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb83f28-6bcf-45d1-9c13-ed753e2c728e service swift] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:22.625338 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb83f28-6bcf-45d1-9c13-ed753e2c728e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:22.634146 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb83f28-6bcf-45d1-9c13-ed753e2c728e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:22.635757 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb83f28-6bcf-45d1-9c13-ed753e2c728e service swift] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:22.643553 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1446/2900] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 23 11:59:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:59:23.456975 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97552fd1-ec06-43e1-ae43-79c8c3febe45 None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:23.457442 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97552fd1-ec06-43e1-ae43-79c8c3febe45 None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:23.457554 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97552fd1-ec06-43e1-ae43-79c8c3febe45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:23.475808 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-97552fd1-ec06-43e1-ae43-79c8c3febe45 None None] MFA Rules not processed for user `030614ef56f143f8be6fcc0445ec9568`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:23.477704 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-97552fd1-ec06-43e1-ae43-79c8c3febe45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:23.485173 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1455/2901] 10.222.0.13 () {58 vars in 967 bytes} [Thu Jun 23 11:59:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:23.497533 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb83f28-6bcf-45d1-9c13-ed753e2c728e service swift] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:23.508251 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb83f28-6bcf-45d1-9c13-ed753e2c728e service swift] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:23.513259 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c65dc-ca06-4eed-8a66-2923b7bf7eae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450b1fad96604fb09d0705895c6d3da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:23.513893 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c65dc-ca06-4eed-8a66-2923b7bf7eae service swift] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:23.514180 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c65dc-ca06-4eed-8a66-2923b7bf7eae service swift] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:23.514458 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c65dc-ca06-4eed-8a66-2923b7bf7eae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:23.522436 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c65dc-ca06-4eed-8a66-2923b7bf7eae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:23.524620 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c65dc-ca06-4eed-8a66-2923b7bf7eae service swift] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:23.532547 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1447/2902] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 23 11:59:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:59:24.218181 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aea47c-e1be-4bdb-aee4-2cfc229f517f None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:24.218463 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aea47c-e1be-4bdb-aee4-2cfc229f517f None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:24.218758 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aea47c-e1be-4bdb-aee4-2cfc229f517f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:24.236207 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-60aea47c-e1be-4bdb-aee4-2cfc229f517f None None] MFA Rules not processed for user `030614ef56f143f8be6fcc0445ec9568`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:24.237383 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-60aea47c-e1be-4bdb-aee4-2cfc229f517f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:24.245102 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1456/2903] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 23 11:59:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:24.256632 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c65dc-ca06-4eed-8a66-2923b7bf7eae service swift] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:24.267958 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c65dc-ca06-4eed-8a66-2923b7bf7eae service swift] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:24.272985 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b2d81-ed44-417b-9af8-d924817a3aad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450b1fad96604fb09d0705895c6d3da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:24.273612 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b2d81-ed44-417b-9af8-d924817a3aad service swift] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:24.273896 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b2d81-ed44-417b-9af8-d924817a3aad service swift] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:24.274224 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b2d81-ed44-417b-9af8-d924817a3aad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:24.282330 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b2d81-ed44-417b-9af8-d924817a3aad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:24.283927 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b2d81-ed44-417b-9af8-d924817a3aad service swift] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:24.292275 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1448/2904] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 23 11:59:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:59:24.908805 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79996ef-9942-4d61-975c-1ebfea412abd None None] REQUEST_METHOD: `POST` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:24.909306 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79996ef-9942-4d61-975c-1ebfea412abd None None] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:24.909424 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79996ef-9942-4d61-975c-1ebfea412abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:24.927112 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.auth.core [None req-f79996ef-9942-4d61-975c-1ebfea412abd None None] MFA Rules not processed for user `030614ef56f143f8be6fcc0445ec9568`. Rule list: `[]` (Enabled: `True`). {{(pid=80255) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:24.928427 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-f79996ef-9942-4d61-975c-1ebfea412abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:24.935605 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1457/2905] 10.222.0.13 () {58 vars in 967 bytes} [Thu Jun 23 11:59:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:24.947596 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b2d81-ed44-417b-9af8-d924817a3aad service swift] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:24.958154 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b2d81-ed44-417b-9af8-d924817a3aad service swift] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:24.963725 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd7bab-5d43-4866-9642-d7e1d1f443e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450b1fad96604fb09d0705895c6d3da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:24.964353 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd7bab-5d43-4866-9642-d7e1d1f443e2 service swift] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:24.964639 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd7bab-5d43-4866-9642-d7e1d1f443e2 service swift] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:24.964917 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd7bab-5d43-4866-9642-d7e1d1f443e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:24.973061 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd7bab-5d43-4866-9642-d7e1d1f443e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:24.974618 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd7bab-5d43-4866-9642-d7e1d1f443e2 service swift] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:24.983447 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1449/2906] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 23 11:59:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:59:27.530408 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79996ef-9942-4d61-975c-1ebfea412abd None None] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:27.592565 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79996ef-9942-4d61-975c-1ebfea412abd None None] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:27.596970 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580293fb-ff0b-484b-a2f1-97f7f9f4e102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:27.597827 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580293fb-ff0b-484b-a2f1-97f7f9f4e102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:27.598109 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580293fb-ff0b-484b-a2f1-97f7f9f4e102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:27.598384 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580293fb-ff0b-484b-a2f1-97f7f9f4e102 admin admin] PATH_INFO: `/v3/users/9153568c8f0b4037afabf05f23da3459` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:27.599579 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580293fb-ff0b-484b-a2f1-97f7f9f4e102 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9153568c8f0b4037afabf05f23da3459)` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:27.601479 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580293fb-ff0b-484b-a2f1-97f7f9f4e102 admin admin] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:27.806377 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-580293fb-ff0b-484b-a2f1-97f7f9f4e102 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9153568c8f0b4037afabf05f23da3459'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:27.813807 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-580293fb-ff0b-484b-a2f1-97f7f9f4e102 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9153568c8f0b4037afabf05f23da3459'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:27.816289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-580293fb-ff0b-484b-a2f1-97f7f9f4e102 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9153568c8f0b4037afabf05f23da3459'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:27.816289 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-580293fb-ff0b-484b-a2f1-97f7f9f4e102 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9153568c8f0b4037afabf05f23da3459'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:27.826919 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.notifications [None req-580293fb-ff0b-484b-a2f1-97f7f9f4e102 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9153568c8f0b4037afabf05f23da3459'} {{(pid=80255) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:27.892680 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1458/2907] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 23 11:59:27 2022] DELETE /identity/v3/users/9153568c8f0b4037afabf05f23da3459 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:59:27.897335 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39c5ef-46f0-4bb2-b540-ab5510a02f92 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:27.897805 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39c5ef-46f0-4bb2-b540-ab5510a02f92 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:27.897908 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39c5ef-46f0-4bb2-b540-ab5510a02f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:27.915823 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-6c39c5ef-46f0-4bb2-b540-ab5510a02f92 None None] MFA Rules not processed for user `d5118162f0624ca794a0f081b0c2cca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:27.919384 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6c39c5ef-46f0-4bb2-b540-ab5510a02f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:27.974408 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1450/2908] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 23 11:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:27.986041 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580293fb-ff0b-484b-a2f1-97f7f9f4e102 admin admin] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:27.987474 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-580293fb-ff0b-484b-a2f1-97f7f9f4e102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:27.993864 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:28.044728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580293fb-ff0b-484b-a2f1-97f7f9f4e102 admin admin] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:28.048320 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b89453-8548-4a34-8c3d-7c181bf91e34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '810fa3f54cd04f989252765f7e76e602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:28.048950 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b89453-8548-4a34-8c3d-7c181bf91e34 service neutron] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:28.049227 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b89453-8548-4a34-8c3d-7c181bf91e34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:28.049494 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b89453-8548-4a34-8c3d-7c181bf91e34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:28.062481 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b89453-8548-4a34-8c3d-7c181bf91e34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:28.062481 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b89453-8548-4a34-8c3d-7c181bf91e34 service neutron] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:28.069728 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1459/2909] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 11:59:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:59:28.357778 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39c5ef-46f0-4bb2-b540-ab5510a02f92 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:28.360817 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6c39c5ef-46f0-4bb2-b540-ab5510a02f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:28.374827 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39c5ef-46f0-4bb2-b540-ab5510a02f92 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:28.380115 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe0fea-cfdd-4d55-9811-077143ef39b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5118162f0624ca794a0f081b0c2cca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '237fd59b9b4946f0a2bdfcf73db9274e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:28.380987 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe0fea-cfdd-4d55-9811-077143ef39b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:28.381267 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe0fea-cfdd-4d55-9811-077143ef39b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:28.381539 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe0fea-cfdd-4d55-9811-077143ef39b7 admin admin] PATH_INFO: `/v3/projects/f3934169b16f4e589b7670d234a38252` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:28.382752 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe0fea-cfdd-4d55-9811-077143ef39b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3934169b16f4e589b7670d234a38252)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:28.386548 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe0fea-cfdd-4d55-9811-077143ef39b7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:28.517398 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:28.520235 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5cbe0fea-cfdd-4d55-9811-077143ef39b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3934169b16f4e589b7670d234a38252', 'request_id': 'req-5cbe0fea-cfdd-4d55-9811-077143ef39b7'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:28.527175 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5cbe0fea-cfdd-4d55-9811-077143ef39b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3934169b16f4e589b7670d234a38252', 'request_id': 'req-5cbe0fea-cfdd-4d55-9811-077143ef39b7'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:28.652936 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5cbe0fea-cfdd-4d55-9811-077143ef39b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3934169b16f4e589b7670d234a38252', 'request_id': 'req-5cbe0fea-cfdd-4d55-9811-077143ef39b7'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:28.663659 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5cbe0fea-cfdd-4d55-9811-077143ef39b7 admin admin] The token cache is being invalidate because project f3934169b16f4e589b7670d234a38252 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:59:28.664534 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5cbe0fea-cfdd-4d55-9811-077143ef39b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:28.664770 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5cbe0fea-cfdd-4d55-9811-077143ef39b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:28.666676 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1451/2910] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 23 11:59:28 2022] DELETE /identity/v3/projects/f3934169b16f4e589b7670d234a38252 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:59:38.081526 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=24 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:38.684836 dv-cinder-820994-2-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:26.851604 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b89453-8548-4a34-8c3d-7c181bf91e34 service neutron] Authenticating user token {{(pid=80255) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:26.853019 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-23b89453-8548-4a34-8c3d-7c181bf91e34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:26.920881 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b89453-8548-4a34-8c3d-7c181bf91e34 service neutron] Validating token access rules against request {{(pid=80255) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:26.978205 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eace74-f226-499d-9cc8-5bcb96b657c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac04b32777ca4f929e9e59deee6e0260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '886c2939a14b48febc2e2fe639c8f1e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80255) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:26.978882 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eace74-f226-499d-9cc8-5bcb96b657c8 service placement] REQUEST_METHOD: `GET` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:26.979171 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eace74-f226-499d-9cc8-5bcb96b657c8 service placement] SCRIPT_NAME: `/identity` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:26.979452 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eace74-f226-499d-9cc8-5bcb96b657c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80255) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:26.981198 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.fernet_utils [None req-a1eace74-f226-499d-9cc8-5bcb96b657c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80255) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:26.993093 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eace74-f226-499d-9cc8-5bcb96b657c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:26.994739 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eace74-f226-499d-9cc8-5bcb96b657c8 service placement] RBAC: Authorization granted {{(pid=80255) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:27.004567 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: [pid: 80255|app: 0|req: 1460/2911] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:03:37.000245 dv-cinder-820994-2-iscsi devstack@keystone.service[80255]: DEBUG dbcounter [-] [80255] Writing DB stats keystone:SELECT=58 {{(pid=80255) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}